Job Description:
We are seeking a skilled Big Data Engineer to join our team and contribute to the design, development, and maintenance of our company's big data platform. The ideal candidate will play a crucial role in ensuring platform stability while meeting business requirements through innovative data solutions.
- Participate in the architectural design, construction, development, and maintenance of the company's big data platform to ensure its stability and performance.
- Design and develop offline and real-time data warehouses based on business needs, utilizing data modeling techniques to build efficient data models that support analytics and decision-making.
- Contribute to the development of data governance platforms, including metadata management and data security initiatives, to enhance data usability and protection.
- Engage in the construction of business data domains (such as spot/contract/wealth management) and indicator systems within the data warehouse, providing valuable analytical insights and recommendations.
- Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions.
- Continuously optimize data processing pipelines and storage solutions for improved efficiency.
Key Responsibilities:
- Develop and maintain scalable data processing systems using Hadoop, Spark, and Flink.
- Implement ETL processes and manage data warehouse technologies including Hive, Presto, and Starrocks.
- Design and implement data models that support business intelligence and analytics needs.
- Ensure data quality and integrity across all data platforms.
- Participate in data governance initiatives to improve data management practices.
Job Requirements:
- Bachelor's degree or higher in Computer Science, Electronic Information, Software Engineering or related fields.
- Minimum 3 years of experience in big data development.
- Proficiency in big data technologies including Hadoop, Spark, and Flink.
- Strong knowledge of data warehouse technologies (ETL, Hive, Presto, Starrocks) and common data modeling approaches.
- Programming skills in at least one language (Java, Python preferred).
- Excellent business acumen with ability to independently analyze and solve business problems.
- Familiarity with data governance, data security, or data quality concepts.
- Experience with AWS or Alibaba Cloud platform components deployment and operation (preferred).
Benefits:
We offer competitive compensation packages and comprehensive benefits. Specific benefits will be discussed during the interview process.
