Strong Skills In Python - Scala Or Java.
4 years experience on full life cycle Big Data production projects.
Experience with AWS services like EMR, Lambda, S3, DynamoDb.
Strong Experience in processing Big Data and analyzing the data using Spark, Map Reduce, Hadoop, Sqoop, Apache Airflow, HDFS, Hive, Zookeeper, etc.
C1X Inc. is a fast-growing, global technology company headquartered in San Jose, US, and with offices in Chennai and Tokyo. Our mission is to simplify and innovate in digital marketing by building unique and large-scale data products. We are a world-class engineering team that encompasses frontend (Angular), backend (API / Java / ), mobile (Android/IOS), and Big Data engineering to deliver compelling products.
You will be a key member of the engineering team, responsible for shaping data products. Youll have the opportunity to work with the C-level staff, business stakeholders, product managers, and engineering teams to define the next-generation customer data management and marketing software.
Responsibilities
- Design and implement distributed data processing pipelines using Spark, Hive, Python, and other tools and languages prevalent in the Hadoop ecosystem on AWS. Ability to design and implement end-to-end solutions.
- Build utilities, user-defined functions, and frameworks to better enable data flow patterns.
- Work with architecture/engineering leads and other teams to ensure quality solutions are implemented, and engineering best practices are defined and adhered to. Create and maintain data documentation and definitions.
- Work with the product owner and scrum master to understand requirements, help the team plan, and execute sprint tickets, work with other technical teams to develop new features.
- Help drive best practices in continuous integration and deliver data quality.
- Manage data flows and set up automation between various data sources.
- Help drive optimization, testing, and tooling of the data products.
Qualifications
- B.E. in Computer Science/Engineering or equivalent.
- Strong demonstrable skills in two of the following programming languages Python, Scala or Java.
- Minimum 4 years experience working on full life cycle Big Data production projects.
- Should have experience with AWS services like EMR, Lambda, S3, DynamoDb.
- Strong Experience in processing Big Data and analyzing the data using Spark, Map Reduce, Hadoop, Sqoop, Apache Airflow, HDFS, Hive, Zookeeper, etc.
- Familiarity with Docker, Airflow, or equivalent data pipeline and workflow management tools, distributed Stream Processing frameworks for Fast & Big Data like ApacheSpark, Flink, Kafka streams.
- Strong skills around developing RESTful APIs.
- Experience with version control using GitHub and Jenkins CI/CD pipelines.
- Intermediate to advanced knowledge of SQL. Relational SQL and NoSQL databases, including Elasticsearch, Postgres, MySql. Experience in SQL tuning, schema design, or analytical programming.
- Experience with different storage formats like Parquet, CSV, and JSON.
- Experience in unit, functional and automated testing.
- Strong expertise in troubleshooting production issues.
- Comfortable working across a wide array of technologies, fast-paced, and results-oriented environment.
- Excellent verbal and writing skills to effectively collaborate with both business and technical teams.
Locations:
Chennai, India
-
Greater Osaka Area C1X Inc.We are seeking a Data Engineer to join our engineering team. · ...
-
Greater Osaka Area C1X Inc.A fast-growing global technology company headquartered in San Jose is looking for a senior backend engineer to develop high-performance servers and integrations with various exchanges. The ideal candidate has experience with JavaScript including ES6 and Nest JS frameworks. · ...
-
Osaka, Osaka Skyworks Solutions, Inc.If you are looking for a challenging and exciting career in the world of technology, then look no further. Skyworks is an innovator of high performance analog semiconductors whose solutions are powering the wireless networking revolution. · ...
-
Osaka Skyworks Solutions, Inc.We are excited about the opportunity to work with you and glad you want to be part of a team of talented individuals who together can change the way the world communicates. · At Skyworks, you will find a fast-paced environment with a strong focus on global collaboration, minimal ...
-
Sr. EHS Engineer 2
3週間前
Osaka, Osaka Skyworks Solutions, Inc.If you are looking for a challenging and exciting career in the world of technology, · Skyworks is an innovator of high performance analog semiconductors whose solutions · are powering the wireless networking revolution. · ...
-
Sr. CAD Engineer 1
2週間前
Osaka, Osaka Skyworks Solutions, Inc.Challenging and exciting career in technology at Skyworks. · ...
-
Senior Data Engineer
2ヶ月前
Osaka AP Advisory Inc. ￥90,000 - ￥180,000 per yearDesign and develop big data platforms, including data warehouses, data lakes, ETL processes, data visualization, and analytics tools. Formulate data processing and storage strategies based on different business requirements. · ...
-
Kyobashi, Tokyo Astellas Pharma ￥5,500,000 - ￥8,500,000 per yearAstellas is undergoing transformation to deliver value for patients. This opportunity involves working on digital transformation with a real impact within a company dedicated to improving lives. · ...
-
Osaka, Osaka Skyworks Solutions, Inc.If you are looking for a challenging and exciting career in the world of technology, then look no further. Skyworks is an innovator of high performance analog semiconductors whose solutions are powering the wireless networking revolution. · ...
-
Osaka, Osaka AgodaWe are seeking highly skilled engineers with a range of experience in fintech to join our team at Agoda. We bridge the world through travel. · ...
-
Osaka, Osaka Amazon Web Services (AWS) ￥150,000 - ￥208,000 per year+We're looking for talented people who want to help us deliver the highest standards for safety and security while providing seemingly infinite capacity at the lowest possible cost for our customers. · +Own as the site SME and POC... · ...
-
Greater Osaka Area Speechify Remote jobWe're looking to hire for our Data side of our AI team at Speechify. This role is responsible for all aspects of data collection to support our model training operations. We are able to build high-quality datasets at petabyte-scale and low cost through a tight integration of infr ...
-
Greater Osaka Area Digital Edge DCWe are looking for an experienced Data Center Operations Engineer to join our fast-growing team in Osaka. As a key member of our operations team, you will be responsible for maintaining the operational readiness of our data center environment on a 24/7 shift basis. You will assis ...
-
Osaka Vantage Data Centers Full timeVantage Data Centers powers cools protects and connects the technology of the worlds well-known hyperscalers cloud providers and large enterprises. · ...
-
Osaka, Osaka Cognizant ￥2,200,000 - ￥3,000,000 per yearWe are seeking a dedicated Sr. Test Analyst with 5 to 7 years of experience in manual testing across various domains particularly in Property & Casualty Insurance. · Conduct comprehensive manual data testing to ensure data integrity and accuracy across systems. · Perform manual a ...
-
Greater Osaka Area C1X Inc.Simplify and innovate in digital marketing by building unique and large-scale data products. · ...
-
Backend Engineer
2日前
Greater Osaka Area C1X Inc.We are a world-class engineering team that encompasses front end (UI), back end (API / Java / ), mobile (Android/iOS), and Big Data engineering to deliver compelling products. · We are looking for a Senior Backend Engineer to join our team. · ...
-
Greater Osaka Area C1X Inc.We are a world-class engineering team that encompasses front end (Angular, ReactJS), back end (API / Java / ), mobile (Android/IOS), and Big Data engineering to deliver compelling products. · ...
-
Product Manager
2日前
Greater Osaka Area C1X Inc.You will be a key member of the product team responsible for shaping futuristic data products working with C-level staff business stakeholders product managers/leads and engineering teams to define the next-generation customer data management and marketing software. ...
-
Osaka AgodaAs an Analyst at Agoda you will identify new partners opportunities cultivate relationship with them deliver best outcome to end customers · ...
-
Greater Osaka Area C1X Inc.We are seeking an experienced Lead UX/UI Designer to shape our data products. You will work closely with C-level staff and engineering teams to define customer data management software. · ...
