Data Engineer - BI Insights Group, Data Platform Department (DPD)

Location: Tokyo, Japan

Department Overview

The Data Platform Department (DPD) at Rakuten Group develops and maintains a comprehensive data platform, empowering over 70 Rakuten services with solutions for data ingestion, discovery, governance, analytics, and querying. We support data-driven decision-making across one of Japan's largest data ecosystems, providing the tools and infrastructure to support key domains such as Data Lakes, Data Warehouses, and Business Intelligence.

Position

Position Details

• ・Hands-on design, development, testing, and implementation of scalable, cloud-based data warehouse solutions, optimized data pipelines, and robust data modeling
• ・Maintain a healthy data warehouse ecosystem through comprehensive infrastructure management (system logs, storage, compute resources, batch scheduling, etc.), ensuring seamless data integration and proactively addressing performance bottlenecks
• ・Internal Tooling: Adapt to and utilize Rakuten’s internal data pipeline orchestration framework, built on top of GCP-managed Airflow
• ・Communication: Collaborate with cross-functional teams to identify data requirements and align them with business objectives
• ・Documentation: Maintain comprehensive documentation, including platform architecture, data dictionaries, infrastructure configurations, operational processes and so on
• ・ Collaborate with Stakeholders: Work closely with end-users to understand requirements, gather feedback, and deliver impactful, customized reports
• ・ Solution Design: Propose and present visualization solutions to relevant stakeholders
• ・ Data Modeling & Mart Design: Architect reporting-specific data marts to ensure high-performance dashboards
• ・ Data Pipeline Management: Develop, maintain, and optimize pipelines, including DAG scheduling, data ingestion, and ETL/ELT processes to ensure data is "collected from" and "delivered to" efficiently and reliably
• ・ Data Quality Assurance: Validate and ensure the accuracy, consistency, and reliability of all data used in visualizations
• ・ Frontend Development: Design, build, and manage interactive reports using modern data visualization tools.

Mandatory Qualifications

• ・ B.S. in Computer Science, or a related field, or equivalent education and experience
• ・ 3+ years of end-to-end experience in data engineering (SQL) and operational experience in cloud-based or Linux systems
• ・ Experience in developing and building robust data warehouse ecosystems using Google Cloud Platform (GCP – BQ, GCS), Python, and shell scripting
• ・ Understanding of data warehouse/big data concepts, data modeling, batch-processing pipelines (ETL), and data integration techniques for managing large volumes of data
• ・ Hands-on experience creating and leveraging data visualization platforms (e.g., DOMO, Tableau, MicroStrategy, PowerBI, etc.)
• ・ Experience with Git

Desired Qualifications

• ・Experience with Atlassian tools (Confluence, Jira, BitBucket)
• ・Experience working on streaming data pipelines using tools such as Airflow, Apache Kafka, or Spark Streaming
• ・Experience managing systems with high Service Level Objectives/Agreements (SLOs/SLAs)
• ・ Effective communication and project management skills
• ・ Quick learner, highly adaptable, and committed to continuous learning
• ・ Strong problem-solving and solution-creation mindset
• ・ Detail-oriented and driven by excellence
• ・ A "Can-Do" attitude and willingness to embrace new challenges
• ・ Strong teamwork and collaboration capabilities
• #engineer #DataEngineer #technologyplatformdiv