Business Overview
AI & Data Division (AIDD) spearheads data science & AI initiatives by leveraging data from Rakuten Group.
We build a platform for large-scale field experimentations using cutting-edge technologies to provide critical insights that enable faster and better and faster contribution for our business.
Our division boasts an international culture created by talented employees from around the world.Following the strategic vision "Rakuten as a data-driven membership company", AIDD is expanding its data & AI related activities across multiple Rakuten Group companies.
Department Overview
GPU Optimization Department (GPUOD) is responsible for the strategic management, optimization, and governance of Rakuten's company-wide AI infrastructure, ensuring high-performance, cost-efficient utilization of compute resources for machine learning workloads.
As a Central Enabler For AI Innovation, We
Optimize compute resource allocation across on-premises and multi-cloud environments, maximizing efficiency for training and inference workloads.
Manage hybrid orchestration of diverse accelerator resources, ensuring seamless scalability and cost-effective deployment.
Develop and enhance frameworks for large-scale distributed training, with special focus on LLMs and generative AI.
Optimize inference performance through model optimization techniques and system-level acceleration.
Collaborate with internal teams to deliver scalable, high-availability inference services tailored to business needs.
Continuously evaluate next-generation hardware solutions, including specialized AI chips optimized for LLM workloads.
By effectively managing both conventional and specialized compute resources across on-premises and cloud environments, our team ensures Rakuten's AI ecosystem remains at the forefront of performance, reliability, and cost-efficiency.
PositionWhy We Hire
Work on cutting-edge LLM training & inference optimization at scale.
Directly impact Rakuten's AI infrastructure by improving efficiency and reducing costs.
Collaborate with global AI/ML teams on high-impact challenges.
Opportunity to research and implement state-of-the-art GPU optimizations.
Position Details
As a GPU Training & Inference Optimization Engineer, you will focus on maximizing the performance, efficiency, and scalability of LLM training and inference workloads on Rakuten's GPU clusters.
You will deeply optimize training frameworks (e.g., PyTorch, DeepSpeed, FSDP) and inference engines (e.g., vLLM, TensorRT-LLM, Triton, SGLang), ensuring Rakuten's AI models run at peak efficiency.
This role requires strong expertise in GPU-accelerated ML frameworks, distributed training, and inference optimization, with a focus on reducing training time, improving GPU utilization, and minimizing inference latency.
Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed, Megatron-LM, FSDP) to maximize GPU utilization and reduce training time.
Profile and optimize distributed training bottlenecks (e.g., NCCL issues, CUDA kernel efficiency, communication overhead).
Implement and tune inference optimizations (e.g., quantization, dynamic batching, KV caching) for low-latency, high-throughput LLM serving (vLLM, TensorRT-LLM, Triton, SGLang).
Collaborate with infrastructure teams to improve GPU cluster scheduling, resource allocation, and fault tolerance for large-scale training jobs.Develop benchmarking tools to measure and improve training throughput, memory efficiency, and inference latency.
Research and apply cutting-edge techniques (e.g., mixture-of-experts, speculative decoding) to optimize LLM performance.
Mandatory Qualifications
3+ years of hands-on experience in GPU-accelerated ML training & inference optimization, preferably for LLMs or large-scale deep learning models.
Strong knowledge of LLM inference optimizations (e.g., quantization, pruning, KV caching, continuous batching).
Bachelor's or higher degree in Computer Science, Engineering, or related field.
Desired Qualifications
Proficiency in CUDA, Triton kernel, NVIDIA tools (Nsight, NCCL), and performance profiling (e.g., PyTorch Profiler, TensorBoard).
Experience with LLM-specific optimizations (e.g., FlashAttention, PagedAttention, LoRA, speculative decoding).
Familiarity with Kubernetes (K8s) for GPU workloads (e.g., KubeFlow, Volcano).
Contributions to open-source ML frameworks (e.g., PyTorch, DeepSpeed, vLLM).
Experience with inference serving frameworks (e.g., vLLM, TensorRT-LLM, Triton, Hugging Face TGI).
#engineer #applicationsengineer #AI #aianddatadiv
Languages
English (Overall - - Advanced)
-
Tokyo RakutenWe are looking for a GPU Training & Inference Optimization Engineer to work on cutting-edge LLM training & inference optimization at scale. · Optimize LLM training frameworks (e.g., PyTorch, DeepSpeed) to maximize GPU utilization and reduce training time. · ...
-
Tokyo Third IntelligenceThird Intelligenceでは、現在主流となっているTransformerアーキテクチャに留まらない、次世代のAIアルゴリズムの研究開発に取り組んでいます。 · An定的かつ継続的にプロダクトへ反映していくために研究と実装の橋渡しを担うResearch Engineerの体制強化を行います。 · ...
-
Tokyo Third IntelligenceThird Intelligence wa, genjitsu ni shite iru Transformer Architcture ni tsuiteyaru, saki no AI Algorithm no kenkyuu kaihatsu ni tsukimashita. · kore nara kenkyuu kekka o jissha sevisu toshite shakai e teiketsushite,お"san-zoku AGI"to ieru konsepu to moto,i kenkyuu kaihatsusei no k ...
-
Tokyo Third IntelligenceThird Intelligenceでは、現在主流となっているTransformerアーキテクチャに留まらない、次世代のAIアルゴリズムの研究開発に取り組んでいます。このような研究成果を、自社サービスとして社会に実装することを見据え、「遍在型AGI」というコンセプトのもと、研究開発体制の強化を進めています。新たなメンバーを募集します。 · ...
-
Tokyo Third IntelligenceThird Intelligenceでは、現在主流となっているTransformerアーキテクチャに留まらない、次世代のAIアルゴリズムの研究開発に取り組んでいます。つまりは、自社サービスとして社会に実装することを目指した「遍在型AGI」というコンセプトのもと、強化を進めています。このため新たにメンバーを募集します。 ...
-
Tokyo SandozThis position is responsible for supporting the management of operational processes in pharmacovigilance to ensure compliance with Sandoz global/local procedures and national/international regulations. · ...
-
Weapons Duty Officer
2週間前
Tokyo Nisga'a Tek, LLCThe Weapons Duty Officer (WDO) will support the PACAF Aviation Squadron Administrative Support for the US Air Force. This position shall act as an assistant to a government security manager. · Maintain accountability of all aircraft media and classified material assigned to the u ...
-
Tokyo Vertiv ￥10,000,000 - ￥20,000,000 per yearImprove operational efficiency to better support Channel partners · Identify structural improvements process improvements and people improvement within the IT Channel Distribution to build a sustainable business · Manage interface between Vertiv and Distributor Manage deal regist ...
-
Tokyo Cornerstone Recruitment JapanThis is a high-impact leadership role where you can truly shape the future of the brand in Japan, influence global strategy, and work closely with international teams—all within a prestigious fashion environment. · ...
-
Tokyo Cushman & Wakefield Full timeWe are seeking a meticulous and organized Document Controller to manage and oversee all document control activities for our data centre projects.Bachelor's degree in Information Management, Business Administration, or a related field. · Minimum of 5 years of experience in documen ...
-
BI Senior Manager
2ヶ月前
Tokyo Red BullThe On Premise Business & Consumer Intelligence Senior Manager is a national role focused on enhancing Red Bull's brand image, market share, · and consumption in the On Premise environment.Create memorable Red Bull experiences that drive word of mouth within On Premise industry c ...
-
In-House Strategy
2週間前
Tokyo Robert Half ￥12,600,000 per yearIn-House Strategy role provides a unique opportunity to join in an impactful transformation initiative within a global organization. Act as the bridge between business and IT teams, collaborate closely with IT, standardize processes, implement governance, and ensure smooth onboar ...
-
Institutional Sales
2ヶ月前
Tokyo GreenstoneWe are hiring an Investor Relations professional focused on Japan. · This is a high-impact, front-office fund placement role responsible for distributing global private market funds to Japanese institutional and professional investors. · ...
-
Ambassador
1ヶ月前
Tokyo Abbott $100,000 - $120,000 per yearRepresents Abbott to targeted customers in order to increase awareness of Abbott and its brand generate sales opportunities and build customer preference Embodies what Abbott stands for and creates awareness attention and sales lead opportunities. · ...
-
Tokyo Specialized GroupA leading global financial services organization is seeking a Project Manager Business Analyst to support its Global Markets CRM platform built on Salesforce. · ...
-
pmo
2週間前
Tokyo Robert HalfLead large-scale digital transformation initiatives across the organization. Drive company-wide transformation that directly impacts strategic goals and competitive positioning. · ...
-
Executive Assistant
1ヶ月前
Tokyo Mastercard ￥3,000,000 - ￥6,000,000 per yearTo support the Japan Division President (Country Leader) · Perform general administrative duties to support day-to-day Division President's operations in such areas as phone coverage; scheduling meetings; completing travel arrangements; expense reporting; · Maintain management ca ...
-
Tokyo Nike Full timeNike's Legal Department seeks an Assistant General Counsel in Japan who combines deep legal expertise with a collaborative mindset to provide counsel and advice to various business functions. · Enable day-to-day commercial matters transactions business projects · Draft support co ...
-
Tokyo HBK - Hottinger Brüel & Kjær Full timeWe are currently looking for a Field Sales Engineer for our team. The position will be based in Tokyo, Japan. · ...
-
Research Scientist
2ヶ月前
Tokyo EnacticBuild Physical AI systems for humanoid robots that assist people in home environments and establish a continuous improvement cycle between real-world data collection and perception-decision-action models. · ...
-
Tokyo Soho House & Cosupport the front office manager and ensure optimal experience for members guests and staff at soho house. · minimum of 3+ years' experience working and 1+ year of fo supervisory experience · ...
.png?1714733948){kind=logo}