Job Description:
Business Overview
AI & Data Division (AIDD) spearheads data science & AI initiatives by leveraging data from Rakuten Group. We build a platform for large-scale field experimentations using cutting-edge technologies to provide critical insights that enable faster and better and faster contribution for our business. Our division boasts an international culture created by talented employees from around the world. Following the strategic vision "Rakuten as a data-driven membership company", AIDD is expanding its data & AI related activities across multiple Rakuten Group companies.
Department Overview
GPU Optimization Department (GPUOD) is responsible for the strategic management, optimization, and governance of Rakuten's company-wide AI infrastructure, ensuring high-performance, cost-efficient utilization of compute resources for machine learning workloads. We oversee a large-scale hybrid infrastructure spanning thousands of accelerators, including the latest Hopper and upcoming Blackwell architectures.
As a central enabler for AI innovation, we:
- Optimize compute resource allocation across on-premises and multi-cloud environments, maximizing efficiency for training and inference workloads.
- Manage hybrid orchestration of diverse accelerator resources, ensuring seamless scalability and cost-effective deployment.
- Develop and enhance frameworks for large-scale distributed training, with special focus on LLMs and generative AI.
- Optimize inference performance through model optimization techniques and system-level acceleration.
- Collaborate with internal teams to deliver scalable, high-availability inference services tailored to business needs.
- Continuously evaluate next-generation hardware solutions, including specialized AI chips optimized for LLM workloads.
- By effectively managing both conventional and specialized compute resources across on-premises and cloud environments, our team ensures Rakuten's AI ecosystem remains at the forefront of performance, reliability, and cost-efficiency.
Position:
Why We Hire
- Work on cutting-edge LLM training & inference optimization at scale.
- Directly impact Rakuten's AI infrastructure by improving efficiency and reducing costs.
- Collaborate with global AI/ML teams on high-impact challenges.
- Opportunity to research and implement state-of-the-art GPU optimizations.
Position Details
As a GPU Training & Inference Optimization Engineer, you will focus on maximizing the performance, efficiency, and scalability of LLM training and inference workloads on Rakuten's GPU clusters. You will deeply optimize training frameworks (e.g., PyTorch, DeepSpeed, FSDP) and inference engines (e.g., vLLM, TensorRT-LLM, Triton, SGLang), ensuring Rakuten's AI models run at peak efficiency.
This role requires strong expertise in GPU-accelerated ML frameworks, distributed training, and inference optimization, with a focus on reducing training time, improving GPU utilization, and minimizing inference latency.
Key Responsibilities
- Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) to maximize GPU utilization and reduce training time.
- Profile and optimize distributed training bottlenecks (e.g., NCCL issues, CUDA kernel efficiency, communication overhead).
- Implement and tune inference optimizations (e.g., quantization, dynamic batching, KV caching) for low-latency, high-throughput LLM serving (vLLM, TensorRT-LLM, Triton, SGLang).
- Collaborate with infrastructure teams to improve GPU cluster scheduling, resource allocation, and fault tolerance for large-scale training jobs.
- Develop benchmarking tools to measure and improve training throughput, memory efficiency, and inference latency.
- Research and apply cutting-edge techniques (e.g., mixture-of-experts, speculative decoding) to optimize LLM performance.
Mandatory Qualifications:
- 3+ years of hands-on experience in GPU-accelerated ML training & inference optimization, preferably for LLMs or large-scale deep learning models.
- Deep expertise in PyTorch, DeepSpeed, FSDP, or Megatron-LM, with experience in distributed training optimizations.
- Strong knowledge of LLM inference optimizations (e.g., quantization, pruning, KV caching, continuous batching).
- Bachelor's or higher degree in Computer Science, Engineering, or related field.
Desired Qualifications:
- Proficiency in CUDA, Triton kernel, NVIDIA tools (Nsight, NCCL), and performance profiling (e.g., PyTorch Profiler, TensorBoard).
- Experience with LLM-specific optimizations (e.g., FlashAttention, PagedAttention, LoRA, speculative decoding).
- Familiarity with Kubernetes (K8s) for GPU workloads (e.g., KubeFlow, Volcano).
- Contributions to open-source ML frameworks (e.g., PyTorch, DeepSpeed, vLLM).
- Experience with inference serving frameworks (e.g., vLLM, TensorRT-LLM, Triton, Hugging Face TGI).
#engineer #applicationsengineer #AI #aianddatadiv
Languages:
English (Overall - 3 - Advanced)-
Distributed Training
1ヶ月前
Tokyo Rakuten Full time￥2,000,000 - ￥2,500,000 per yearAs a GPU Training & Inference Optimization Engineer, you will focus on maximizing the performance, efficiency, and scalability of LLM training and inference workloads on Rakuten's GPU clusters. · Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) to ma ...
-
Distributed Training
1ヶ月前
Tokyo Rakuten ￥5,000,000 - ￥10,000,000 per yearAs a GPU Training & Inference Optimization Engineer, you will focus on maximizing the performance, efficiency, and scalability of LLM training and inference workloads on Rakuten's GPU clusters. · Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) to ma ...
-
Tokyo Michael Page ￥6,000,000 - ￥10,000,000 per yearBuild your career in a global tech leader working with advanced semiconductors Benefit from competitive pay and a performance-driven culture · Develop and execute sales strategies to achieve distribution targets · Build and maintain strong relationships with distributors and key ...
-
Trade Manager
2ヶ月前
Tokyo Michael Page ￥900,000 - ￥1,200,000 per yearOpportunity to lead strategic trade initiatives Clear career advancement pathways · Develop and implement trade strategies to maximize revenue and market share. · Analyze market trends and competitor activities to identify growth opportunities. · Manage pricing, negotiation, and ...
-
Tokyo Telexistence Inc ￥5,500,000 - ￥10,000,000 per yearAs the Robotics Foundation Model Engineer, you will design, train, and deploy large-scale multimodal models that integrate vision, language, and action components for real-world robotic applications. · Model Architecture & Implementation · Design Vision-Language-Action Models · S ...
-
MLOps Engineer
1ヶ月前
Tokyo AI Robot Association Full time￥4,000,000 - ￥12,000,000 per yearWe are launching a groundbreaking initiative: collecting one million hours of humanoid robot operation data with hundreds of robots, and leveraging it to train the world's most powerful Vision-Language-Action (VLA) models. · Design, implement, and maintain large-scale ML pipeline ...
-
Solutions Architect
1ヶ月前
Tokyo E-Solutions ￥10,000,000 - ￥20,000,000 per yearSeeking a hands-on SAP Sales & Distribution (SD) professional with strong leadership, communication, and cross-functional collaboration skills. · Lead SAP SD (Order-to-Cash) workshops and solution design sessions. · Analyze business processes, recommend SAP best practices, and as ...
-
Data Engineer
1ヶ月前
Tokyo AI Robot Association Full time￥4,000,000 - ￥12,000,000 per yearPlay a critical role in building the data backbone powering next-generation robotics foundation models. · ...
-
Executive Assistant
1ヶ月前
Tokyo Mastercard ￥3,000,000 - ￥6,000,000 per yearSupport the Japan Division President and the entire Japan Leadership Team in their day-to-day operations, including telephone coverage, scheduling meetings, travel arrangements, expense reporting, and more. · Perform general administrative duties to support day-to-day Division Pr ...
-
Executive Assistant
1ヶ月前
Tokyo Mastercard Full time￥4,000,000 - ￥6,000,000 per yearSupport the Japan Division President and Japan Leadership Team with administrative duties, including scheduling, travel arrangements, and correspondence. · Perform general administrative duties to support day-to-day Division President's operations · Prepare correspondence, memora ...
-
Tokyo Oracle Japan 日本オラクル ￥4,000,000 - ￥8,000,000 per yearWe are seeking a Cloud Support Engineer to provide exceptional customer support tailored to our enterprise clients in Japan. You will collaborate closely with internal teams in English and directly with customers in Japanese, focusing on troubleshooting complex technical issues a ...
-
Tokyo Amazon Web Services (AWS) ￥1,200,000 - ￥1,500,000 per yearTo meet the growing demand for Amazon Web Services around the globe, we need exceptionally talented, bright, and driven people. We are looking for a dynamic, organized self-starter to join our APJ Training & Certification Team. · Develop and execute partner sales & delivery plans ...
-
Tokyo iSearchJoin a world-leading organization with over 157,000 employees in 200 countries and build an international career by joining a world-leading organization with over 157,000 employees in 200 countries. · Manage and grow relationships with brokers, agents, and Fortune 500 companies t ...
-
Legal Specialist
2ヶ月前
Tokyo EssilorLuxottica ￥900,000 - ￥1,200,000 per yearWe are seeking a Legal Specialist to act as Company Secretary of all Board of Directors and AGM/EGMs / shareholders meetings, including draft, review and coordination of signatures of call notices, agendas, supporting legal and business documents, Minutes, books and records of th ...
-
Tokyo NBCUniversal Full time￥10,000,000 - ￥20,000,000 per yearWe are seeking an experienced business affairs executive with legal experience to oversee the Legal & Business Affairs function in our Tokyo office. The successful candidate will provide commercially focused business affairs and legal support within a media company or studio, inc ...
-
HEOR Specialist
2ヶ月前
Tokyo Barrington James ￥3,000,000 - ￥4,500,000 per yearProvide project-related support and assistance for assigned clinical project(s) and project teams, operating with minimal supervision. · Bachelor's Degree in life sciences or a related field is required. · Typically requires 0-2 years of prior relevant experience. · Foundational ...
-
Tokyo SMALL WORLD / Work in Japan? ￥8,000,000 - ￥15,000,000 per yearThis is a Contract Position for Rakuten. As the AI Engineer for our innovative tech company, you will play a pivotal role in shaping the future of our AI-driven solutions. · Designing, developing, and deploying advanced AI models, with an initial emphasis on leveraging Large Lang ...
-
Tokyo Prada Group ￥600,000 - ￥1,800,000 per yearThe Merchandising Manager is responsible for overseeing purchasing activities in Japan and the Islands. This role involves managing merchandising activities—including buying, new product launch strategies, network and distribution management, and markdown planning—to ensure optim ...
-
Tokyo Barco ￥5,000,000 - ￥10,000,000 per yearDevelop and execute comprehensive distribution and channel management strategies to optimize market coverage and maximize sales potential. Prepare and manage sales forecasts and reports in alignment with business objectives. Develop, manage, and present quarterly business reviews ...
-
Tokyo FICO ￥4,320,000 - ￥10,800,000 per yearPartner Account Managers combine industry knowledge, senior relationship building skills and a solutions-based, consultative-selling approach to expand the number of FICO clients serviced across Japan via our channel partners. · Leverages the above details, and knowledge of FICO' ...
-
Tokyo Jobs via eFinancialCareers ￥1,200,000 - ￥2,400,000 per yearJoin us at DWS, and you can be part of an industry-leading firm with a global presence. You can lead ambitious opportunities and shape the future of investing. You can support our clients, local communities, and the environment. · Work closely with generalist sales professionals ...
