Big Data Developer
ID: #12167 Date: 14/06/17
|Location||England, South East, Surrey, Elmbridge, Cobham and Downside|
|Sector(s):||IT Contractor jobs|
Essential skills and knowledge
- Proficient in Python and SPARK development
- Well experienced working on Hadoop environments with various suite of services [Oozie, HIVE, Impala, ELK, Kafka, Storm, Sqoop etc]
- Experience working with various formats – XML, JSON,AVRO, PARQUET etc.
- Strong experience in Data Ingestion and processing on Big Data
- ETL and Data Warehousing using Microsoft BI Stack
- Strong self-sufficiency and initiative working on Business Intelligence projects
- Performance tuning and best practices on Big Data projects
- Python [REST API], Pyspark
- SPARK and MapReduce
- oozie, HIVE, Impala, ELK
- Yarn, Shell Scripting
- Able to structure, organise and optimise large amounts of data
- Deployment automation
- MS SQL Server 2008 r2/2012/2014 product stack
- SPs, functions and data manipulation with SSIS
A very hands-on specialist technical role to support and improve AXA Insurance UK data systems under the guidance of team leads and the data solution architects. As a member of Data Services you will be supporting and enhancing the current Big data and Data Warehouse environment. The team member will be collaborating with other Data Services teams and other business as well as IT teams. As a Big data developer with ETL skills, should be able to ingest and process data on Hadoop as well as provide data feeds where required. You will be the go-to person within the team for Python development, Spark, Hadoop ecosystem, Cloudera specific tools like Cloudera Navigator and Cloudera Manager, T-SQL and SSIS. Main Duties & Responsibilities
- Develop and support the Cloudera Hadoop implementations in Data Services.
- Good knowledge of testing in a Big data environment.
- Developing, testing and implementing components for the existing data solutions using industry best practice techniques.
- Excellent coding and deployment practice.
- To work as part of a team assisting and supporting other team members.
- To provide support and assistance to other members and departments of the business as needed. This includes, but is not limited to software developers, underwriting, software testing, marketing and management.
- Troubleshooting, debugging and improving existing processes.
- Produce technical and user documentation for processes and procedures.
Tell about this vacancy friends
Phone 020 3206 8300