Senior Databricks & Spark Consultant

---

Job description

Job Description

Lovelytics is seeking a creative, technical, and entrepreneurial Senior Databricks and Spark Consultant to be one of the leading data minds at Lovelytics.

The Senior Databricks and Spark Consultant will lead engagements with strategic clients related to ML operations, ETL pipeline optimization, and data architecture. In this role, you will be deploying industry-leading technologies such as Databricks, Spark, and MLflow.

Role Location: Arlington, VA or Toronto, ON

Primary Job Responsibilities

* Fill the Solution Architect role on strategic engagements with clients focused on developing data strategy, and optimizing back-end performance.
* Accelerate and enable our Data Scientists with engineering and automation support.
* Maintain relationships with strategic vendor partners such as Databricks and bring technology thought leadership to the client site.
* Gather requirements from clients and develop creative and effective technical solutions.
* Oversee and grow the skillset of a team of consultants.
* Manage projects to ensure project milestones are reached within the given timeline and budget allocated.
* Extract insights from data provided by clients and provide visually-appealing and high-performing reports, dashboards, and automated services.
* Work with agility and wear many hats in a scrappy, entrepreneurial environment.
* Troubleshoot highly technical, large-scale data issues on the fly with prospects and clients.
* Provide suggestions to constantly improve internal processes and methodologies.
* Assist with business development opportunities with prospective clients.

What We Promise You

* Exciting projects with great clients in varying departments and verticals across the world.
* The ability to work closely with all sizes of companies, ranging from Fortune 100 to small local businesses.
* A workplace where you are encouraged to challenge the status quo and develop new technologies, methodologies, and processes.
* A diverse team consisting of data gurus, experience seekers, and entrepreneurial minds that are always pushing to be better.

What Makes Us Excited About a Candidate

* You have a positive attitude coupled with excellent communication skills.
* Your entrepreneurial spirit pushes you to find creative solutions at Lovelytics and on the client site..
* You are motivated by providing superior solutions and disrupting the status quo.
* You have a willingness to travel to customer sites if required.
* You are not afraid to voice your ideas about improving methodologies and processes.
* You thrive in the excitement of a fast-growing company.
* You are great with people and enjoy leading a technical team.

Requirements

Our Ideal Candidates Skills and Experiences

* Significant experience gathering client requirements and translating them into technical and system specifications.
* Familiarity using Databricks or other data analytics platforms
* Experience as a solution architect for business information systems, focusing on database architecture, data modeling, data analysis, and application integration.
* Experience developing and optimizing spark pipelines (Python & SQL).
* Familiarity with the Data Science workflow - what is a model, how is it promoted into production, how are models evaluated, etc.
* Experience with relational and dimensional data modeling using ERWIN, understanding of star and snowflake schemas, familiarity with feature engineering pipelines, and thorough understanding of data warehousing concepts like facts, dimensions, and surrogate keys.
* Strong hands-on knowledge with application development, data platforms, and databases (Databricks, Oracle, SQL Server, Redshift, Snowflake, Postgres, BigQuery), application software SDLC and business reporting/analytics.
* In-depth knowledge of large database design techniques, data transformation, and migration using ETL tools.
* Experience in consulting or delivering projects for external stakeholders.
* Familiarity with Tableau and other business intelligence tools.

Benefits

Some of Our Benefits

* Unlimited paid time off with an annual minimum amount of days employees must use.
* 401K with matching.
* Transportation benefits, including Arlington Bike Share subscription and pre-tax travel reimbursements.
* Medical, dental, life, and other insurance coverages.
* Frequent out-of-office company outings.
* Quarterly company bonuses based on company performance.
* Annual company bonuses