Job Description:
- Design and develop big data platforms, including data warehouses, data lakes, ETL processes, data visualization, and analytics tools.
- Formulate data processing and storage strategies based on different business requirements, ensuring data efficiency and reliability.
- Write high-quality code to implement data cleaning, transformation, loading, and processing, ensuring data accuracy and integrity.
- Collaborate closely with data analysts, business personnel, and other technical teams to provide data support and guidance.
- Continuously optimize the performance and reliability of the data platform, effectively tuning and optimizing for large-scale data processing.
- Keep up with the latest big data technologies and trends, continuously learning and improving technical skills.
Job Requirements:
- Bachelor's degree or above in Computer Science, Mathematics, Physics, or related fields.
- Over 5 years of experience in big data platform development, including proficiency in Hadoop, Spark, Hive, Flink, Clickhouse, and other big data technologies and frameworks.
- Proficient in Java programming, with strong coding skills and awareness of code quality.
- Familiar with big data platform development technologies such as data warehouses, data lakes, ETL processes, data visualization, and analytics tools.
- Strong communication and teamwork skills, able to collaborate with people from different backgrounds, and explain complex technical issues to both business and technical personnel.
- Strong problem-solving skills and self-motivation, capable of independent thinking and solving problems.
- Experience in server-side development; full-stack development skills are a plus.
职位描述
1、设计和开发大数据平台,包括数据仓库、数据湖、ETL流程、数据可视化和分析工具等。
2、针对不同的业务需求,制定数据处理和存储策略,确保数据的高效和可靠。
3、编写高质量的代码,实现数据的清洗、转换、加载和处理,保证数据的准确性和完整性。
4、与数据分析、业务人员和其他技术团队紧密合作,为他们提供数据支持和指导。
5、不断优化数据平台的性能和可靠性,处理大规模数据时能够有效地调优和优化。
6、掌握最新的大数据技术和发展趋势,不断学习和提高自己的技术水平。
职位要求
1、本科及以上学历,计算机、数学、物理等相关专业。
2、5年以上大数据平台开发经验,包括但不限于掌握Hadoop、Spark、Hive、Flink、Clickhouse等大数据技术和框架。
3、熟练使用Java 编程语言,有良好的编程能力和代码质量意识。
4、了解数据仓库、数据湖、ETL流程、数据可视化和分析工具等大数据平台开发相关技术。
5、具备良好的沟通能力和团队合作精神,能够与不同背景的人合作,并能够向业务人员和技术人员解释复杂的技术问题。
6、具备良好的问题解决能力和自我驱动力,能够独立思考和解决问题。
7、有服务器开发经验,全栈开发能力更佳。
-
Greater Osaka Area Randstad Japan特に初期臨床開発フェーズにおけるデータマネジメント活動の主要な専門家としての役割を担います。主な責務には、ベンダーマネジメント、スタッフの育成などが含まれます。 · ...
-
Senior Data Engineer
1ヶ月前
Osaka AP Advisory Inc. ￥90,000 - ￥180,000 per yearDesign and develop big data platforms, including data warehouses, data lakes, ETL processes, data visualization, and analytics tools. Formulate data processing and storage strategies based on different business requirements. · ...
-
Osaka, Osaka AgodaWe are seeking highly skilled engineers with a range of experience in fintech to join our team at Agoda. We bridge the world through travel. · ...
-
Ibaraki 日本ポール製造(株) ￥8,050,000 - ￥9,200,000 per year「茨城」Senior Sustaining Engineer」のポジションの求人です製造技術エンジニア(プロジェクトエンジニア)として主に営業利益率拡大に関連する戦略的プロジェクトの計画・主導を行い、変更管理と優れた問題解決原則を活用する責任を負います。主に消耗品製造分野において、多面的なチームと緊密に連携し、プロセス改善に注力します。 · ...
-
Kyobashi, Tokyo Astellas Pharma ￥5,500,000 - ￥8,500,000 per yearAstellas is undergoing transformation to deliver value for patients. This opportunity involves working on digital transformation with a real impact within a company dedicated to improving lives. · ...
-
Osaka, Osaka Amazon Web Services (AWS) ￥150,000 - ￥208,000 per year+We're looking for talented people who want to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. · +Own as the site SME and POC... · ...
-
Greater Osaka Area Speechify Remote jobWe're looking to hire for our Data side of our AI team at Speechify. This role is responsible for all aspects of data collection to support our model training operations. We are able to build high-quality datasets at petabyte-scale and low cost through a tight integration of infr ...
-
Osaka, Osaka Bayer ￥3,240,000 - ￥8,320,000 per yearTheDataManagementExpert servesasakeysubjectmatterexpertindatamanagementactivities,paticularlyinearlyclinicaldevelopmentphases. · DataManagementExpertise · Actastheprimaryexpertondatamanagementprocessesinearlyclinicaldevelopment · ...
-
Osaka, Osaka TalentPop App Remote job$41,600 - $624,000 per yearThis is a part-time, fully remote contractor role where you'll be the bridge between our clients and our engineering team. You'll join calls, clarify what needs to be built, and keep non-technical stakeholders confidently in the loop. ...
-
Osaka, Osaka Skyworks Solutions, Inc. ￥1,815,500 - ￥2,422,000 per yearWe will find a fast-paced environment with a strong focus on global collaboration, · We are excited about the opportunity to work with you and glad you want to be part of a team of talented individuals who together can change the way the world communicates. · ...
-
Osaka, Osaka Agoda ￥3,500,000 - ￥10,000,000 per yearWe bridge the world through travel at Agoda. · Develop performance-critical applications using Scala or equivalent modern programming languages. · Design scalable architectures · ...
-
JP - Osaka 54 MTJKK ￥5,000,000 - ￥10,000,000 per yearJob summary · Affect the arc of individual lives and lift the trajectory of entire industries at Marvell by developing customized NVMe storage products for specialized use cases. · Responsibilities · Architect design and develop system software for reliable scalable high performi ...
-
Osaka, Osaka Marvell Technology ￥7,000,000 - ￥12,000,000 per yearSenior Staff Engineer position at Marvell in Osaka, Japan. Develop customized NVMe storage products using modern languages like Rust. · ...
-
Osaka, Japan Bayer Crop ScienceThe Senior/Principal Clinical Data Manager serves as a key subject matter expert in data management activities, particularly in early clinical development phases. · ...
-
Greater Osaka Area HCLTech $100,000 - $120,000 per year++This role is ideal for someone who can act as a technical bridge between onsite teams and offshore development members. · + ...
-
Osaka Tanium Full timeWe're searching for a business value professional to collaborate with a high-performing team of sales professionals. · This role requires an intellectually curious individual with keen business and financial acumen. · The ideal candidate will articulate how the Tanium platform ca ...
-
Osaka, Osaka TaniumWe are searching for a business value professional to collaborate with a high-performing team of sales professionals. · ...
-
Osaka, Osaka Agoda ￥960,000 - ￥1,296,000 per yearAbout Agoda · At Agoda...data monitoring tools. · ...
-
Greater Osaka Area Digital Edge DCWe are looking for an experienced Data Center Operations Engineer to join our fast-growing team in Osaka. As a key member of our operations team, you will be responsible for maintaining the operational readiness of our data center environment on a 24/7 shift basis. You will assis ...
-
Osaka Speechify Full timeThe mission of Speechify is to make sure that reading is never a barrier to learning. · Proven experience in backend development: Python · Direct experience with GCP and knowledge of AWS, Azure, or other cloud providers. · ...
-
Osaka, Osaka Agoda $90,000 - $120,000 per yearWe are looking for an experienced MLOps Engineer to join our team at Agoda. · A successful candidate will have experience in designing and building MLOps infrastructure, · including data pipelines, model training and deployment systems · ,and monitoring tools.We bridge the world ...
