Data Engineer (210003HO) Data Engineer (210003HO) …

in Puteaux, Ile-de-France
Permanent, Full time
Last application, 14 Oct 21
in Puteaux, Ile-de-France
Permanent, Full time
Last application, 14 Oct 21

As a data engineer, you will join the Data Platform team responsible for designing, implementing and running cloud-native data solutions. You will either integrate the core team of 4 experienced data engineers or a 2 to 5 product oriented engineering/data science squad of data engineers using the platform daily to service business needs on specialized use cases such as ESG, Data Science, Signals, Risk, Quantitative Data Sourcing, Data Referentials… 

As a Data Engineer, you will have to:
•    port the existing use cases and processes from our on premise legacy hadoop-based plaftorm to the Azure Cloud, using state-of-the art technologies (Databricks, Spark 3, Azure Cognitive Services, Azure Event Hub, Docker, Azure Pipelines, Azure Data Factory, Scala, Python)
•    help with the onboarding of new product squads directly on the new cloud Data Platform
•    gather engineering requirements and patterns common to data use cases and data squads
•    design, build and maintain common patterns such as CI/CD Pipelines, shared libraries (data pipeline development, data quality, data lineage) and shared services (REST API, data viz, monitoring, scheduler),
•    support a community of data engineers and data scientists by understanding their problems and answering their questions and help them write the solutions on the Data Platform
•    participate to the build of our Data Science platform
•    participate to the data onboarding of third-party data providers such as Bloomberg or internal applications
•    design and maintain APIs
•    build a research environment for our Quants
You will get better understanding of the development cycle of/on big data platform shared by many teams and learn how to find efficient solutions to their common challenges




Education / Qualifications / Key experiences

•    Master’s degree, in Computer Science, Engineering, Mathematics or a related field
•    Hands on experience leading large-scale global data warehousing and analytics projects
•    5+ years of experience of implementation and tuning of Data Lake/Hadoop Spark platforms, BI, ETL, …
•    Experience in defining, implementing and scaling Data Modelling or API Design practices
•    Experience delivering data platforms in Azure Cloud
•    Strong experience in the design and implementation of several of the following:
-    Master & Reference Data Management
-    Metadata Management
-    Data Quality Management
-    Data Analytics and BI
-    Data Modelling
-    Data Exchanges
•    English - Fluent in speaking and writing
Technical competences
•    Spark (preferably on Databricks)
•    Scala or Python (preferably both)
•    Cloud computing practices: Infra as Code, security (preferably on Azure)
•    Experience working with data of various formats (np. Avro, Parquet, JSON, CSV)
•    Experience in designing and building APIs + API management is a plus
•    Git + CD/CI

Optional technical skills for a Data Engineer 

•    Azure Cloud – Kubernetes, DataFactory, Azure functions, Cognitive Services, Event Hub, Pureview, Webapp
•    Docker
•    Azure DevOps
•    Data Bitemporal experience

More Jobs Like This
See more jobs