Ma, Xiaoqi

Data Engineer / Software Engineer

Basic Information


  • Aug 1994
  • Hangzhou, China
  • Master of Science

Contact Information


Skills


Languages


  • Native
  • Fluent
  • A2

WORK EXPERIENCE


  • Huawei Enterprises Telecommunication Technologies Co., Ltd

    Data Engineer

    • Served as a Data Engineer & Data Analyst
    • Over 3 years of experience in data engineering, machine learning, and data warehouse
    • Rich experience in Python (as well as Java), SQL, ML algorithms, Big Data ecosystems (i.e. Hadoop, Hive, Spark)

EDUCATION


  • RWTH Aachen University

    Media Informatics, Master

  • Technische Hochschule Lübeck-Information Technology

    Informatics

  • East China University of Science and Technology (ECUST)

    Computer Science

PROJECT EXPERIENCE


  • OmniAdvisor - Automated HyperParameter Tuning

    • Orchestrated the entire design and development of OmniAdvisor, an intelligent system for end-to-end parameter tuning of Spark and Hive tasks within online systems, leveraging AI algorithms
    • Accomplished data preprocessing, data cleansing and other data workflows
    • Constructed ML pipelines, utilizing CNN, Random Forest algorithms
  • OmniMV - Intelligent Recommendation System for Materialized Views

    • Designed and implemented ETL pipelines, targeting at Spark/ Clickhouse/PostgreSQL
    • Generated materialized views utilizing ML algorithms (e.g. xgboost) to enhance SQL execution efficiency
    • Rewritten SQL physical plan to improve query execution efficiency
  • Data Analysis and Performance Optimization

    • Hardware statistics analysis, such as CPU utilization, disk I/O usage, memory usage and network bottlenecks
    • Performance bottleneck analysis for Big data components, such as Spark, Hive, Hadoop
    • Hotspot function identification and optimization
    • Fine-Tuning for Software, e.g. Spark, Hive, Clickhouse, Presto and etc.

© 2023 Xiaoqi Ma. Last update: .