Job Description
We are seeking a talented individual to join our team as a Distributed Crawler System Developer. In this role, you will be responsible for designing and developing efficient, scalable distributed crawler systems for intelligent collection and aggregation of massive social media data across multiple platforms. You'll have the opportunity to work on cutting-edge AI applications, including dynamic lexicon optimization for trending events and precision message push strategies based on user profiling and content understanding. Additionally, you'll explore innovative ways to apply AI technologies in our data processing pipeline, covering areas such as sentiment analysis, hot event identification, and trend prediction. The position also involves learning and practicing big data processing techniques, including data cleaning, analysis, and visualization to uncover valuable insights from complex datasets.
Key Responsibilities
- Design and develop high-performance, scalable distributed crawler systems for massive social media data collection
- Implement and optimize dynamic lexicon systems for trending event tracking
- Develop precision message push strategies based on user profiling and content analysis
- Apply AI technologies in data processing workflows (sentiment analysis, event detection, trend prediction)
- Perform big data processing, cleaning, analysis and visualization to extract valuable insights
- Continuously explore and implement innovative technical solutions
Job Requirements
- Current undergraduate or graduate student in Computer Science, Software Engineering, Data Science or related fields (juniors, seniors, or first/second year graduate students preferred)
- Strong passion for technology with desire to deepen expertise in Python and related technologies
- Willingness to take on challenges and explore new technologies
- Demonstrated sense of responsibility and proactive work attitude
- Excellent teamwork and collaboration skills
- Basic understanding of distributed systems and data processing is a plus
- Familiarity with machine learning concepts and applications is desirable
Benefits & Perks
- Weekends off plus all national holidays
- Company-provided iMac workstations
- Annual team building trips and activities
- Comprehensive social insurance and housing fund
- Year-end bonuses and performance incentives
- Opportunities for professional growth and skill development
- Dynamic and innovative work environment
