Job title: Data Engineer
Location: UK
Duration: 6-month contract Inside IR35
Working Mode: Fully Remote
Vetting: Active SC Clearance
Job Description:
Data Engineer Skills and Technologies Data Engineering:
– Strong understanding of data concepts
– data types, data structures, schemas (both JSON and Spark), schema management etc
– Strong understanding of complex JSON manipulation – Experience working with Data Pipelines using a custom Python/PySpark frameworks
– Strong understanding of the 4 core Data categories (Reference, Master, Transactional, Freeform) and the implications of each, particularly managing/handling Reference Data.
– Strong understanding of Data Security principles
– data owners, access controls
– row and column level, GDPR etc including experience of handling sensitive datasets
– Strong problem solving and analytical skills, particularly able to demonstrate these intuitively
Requirements
Required Skills:
Languages / Frameworks
– JSON
– YAML
– Python (as a programming language, not just able to write basic scripts. Pydantic experience would be a bonus.)
– SQL
– PySpark
– Delta Lake
– Bash (both CLI usage and scripting)
– Git
– Markdown
– Scala (bonus, not compulsory)
– Azure SQL Server as a HIVE Metastore (bonus) Technologies
– Azure Databricks
– Apache Spark
– Delta Tables
– Data processing with Python
– PowerBI (Integration / Data Ingestion)
– JIRA
...The Overbrook School for the Blind is seeking a Public Relations and Marketing Specialist who exudes a passion for the school, its students... ...of a private non-profit school for ages 3 to 21. Formal and informal parental feedback consistently highlights the caring and...
Wurzak Hotel Group is looking for an experienced Steward/Dishwasher to support the Kitchen at our gorgeous property, DoubleTree located in Deerfield Beach, FL. This position will be responsible for cleaning and sanitizing dishes so that Cooks and Servers have the supplies...
...developed and empowered to fulfil their aspirations. Join Gilead and help create possible, together.**Job Description****Director** **, Talent Acquisition:** **Global Employer Brand** **& Talent Attraction Lead**The Company:At Gilead, we're creating a healthier world for...
...Brewery Production Technician Legally Highest (Chattanooga, TN) Full-Time & Part-Time| Must Have Prior Brewery Experience Legally Highest, Tennessees leading Hemp seltzer brewery, is hiring an experienced Brewery Production Technician to join our manufacturing team...
...DescriptionJoin a dynamic bath remodel business and work closely with the senior production manager to ensure smooth operations. As a production assistant, you will be responsible for scheduling installations and maintaining a well-organized schedule, anticipating tasks...