As a Junior Data Engineer, you will play a crucial role in the development, deployment, and maintenance of observability and monitoring solutions deployed alongside High Performance Computing (HPC) clusters and / or Generative Artificial Intelligence (GenAI) platforms.
Observability and data engineering of metrics collected from HPC and AI clusters are crucial for understanding performance bottlenecks, forecasting energy costs and carbon intensity, and predicting anomalies. Using modern data collection tools and ML / Methods enables the creation of Operational Data Analytics (ODA) pipelines and the development of Digital Twins of clusters.
This role requires expertise in basic ETL (Extract / Transform / Load) workflows and the technologies that underpin them. It involves an approximately equal mix of R&D work to develop solutions and support / deploy tasks to deliver and support customers, combining technical skills and customer interaction. The R&D work includes expanding and documenting the ExaMon solution for HPC :
Key Responsibilities :
* Solution Development : Enhance the existing codebase, support new CPU or accelerator architectures, fix bugs, improve usability, and add features in collaboration with the Team Leader and Senior Developers.
* Troubleshooting : Identify and resolve issues related to hardware, software, and network connectivity.
* Documentation : Create and maintain documentation related to deployment, operations, configurations, and delivery processes.
* Client Interaction : Work directly with clients to understand their needs, carry out proof of concept analyses, and provide technical support during and after installation.
* Collaboration : Collaborate with the R&D team to investigate the performance and reliability of new hardware being validated in the laboratory.
Qualifications :
* Bachelor’s degree in Computer Science, Information Technology, Information Engineering, or related STEM fields, or equivalent field experience.
Required Skills :
* Proficiency in Python programming.
* Understanding of Linux / Unix environments.
* Fundamental knowledge of SQL and database concepts.
* Knowledge of REST APIs and HTTP methods.
* Experience with Git version control.
* Basic understanding of container runtimes like Docker and Podman.
* Basic knowledge of ETL processes.
Preferred Skills :
* Basic knowledge of Apache projects like Cassandra, Trino, or Spark.
* Basic understanding of IoT messaging protocols.
* Basic knowledge of NoSQL and / or time series databases.
Additional Skills :
* Experience with data processing frameworks (e.g., Pandas, PySpark).
* Basic understanding of ML / DL / AI frameworks (scikit-learn, PyTorch).
* Knowledge of data warehousing concepts.
* Familiarity with other programming languages (Go, C, C++).
* Experience with data visualization tools (Grafana).
J-18808-Ljbffr
#J-18808-Ljbffr