Databricks Engineer Job at AI Talent Flow, New York, NY

WFpYS21GSE9iSmhsQzNzckFMbjQxeUZBK2c9PQ==
  • AI Talent Flow
  • New York, NY

Job Description

Job Description

Job Description

AI TalentFlow: Databricks Engineer

About AI TalentFlow:

AI TalentFlow specializes in accelerating startup growth, scaling enterprise solutions, and nurturing the career growth of the individuals who power those organizations. Our success is intertwined with your success, and we are committed to fueling the innovative spirit of our clients with top-tier talent in AI, Data, Analytics, Product, and Agile.

The Opportunity:

Play a critical role in leveraging data to drive innovation within the financial services sector! You will collaborate with business stakeholders, data scientists, and software engineers to design, develop, and implement efficient and scalable data pipelines on Databricks. You will also be responsible for identifying and building valuable use cases for data within the financial services domain (e.g., trading, broker-dealer, asset management, investment management). This role offers the chance to make a real impact by contributing to the development of robust and repeatable data-driven solutions that deliver tangible business value.

Responsibilities:

  • Partner with business stakeholders to understand their data needs and identify high-impact use cases for data within the financial services industry.
  • Design, develop, and implement efficient and scalable data pipelines using Apache Spark on Databricks.
  • Develop and maintain data pipelines for various financial services domains, such as trading, risk management, portfolio analysis, and regulatory compliance.
  • Perform data cleaning and transformation tasks to ensure data quality and consistency.
  • Build robust and reusable data pipelines that adhere to best practices and coding standards.
  • Collaborate with data scientists to prepare data for machine learning models.
  • Develop and integrate Python scripts and SQL queries for data manipulation and analysis.
  • Leverage cloud platforms like AWS and Snowflake for data storage and integration.
  • Monitor and maintain data pipelines to ensure smooth operation and identify potential issues.
  • Stay up-to-date on the latest advancements in data engineering technologies and financial services trends.
  • Document data pipelines and processes for clear communication and knowledge sharing.

Qualifications:

  • 3+ years of experience as a Data Engineer or related role.
  • Strong understanding of data warehousing concepts and data modeling principles.
  • Extensive experience with Apache Spark and Databricks (or similar data processing frameworks).
  • Proficiency in Python and SQL, with experience in data manipulation and analysis libraries.
  • Experience building data pipelines for the financial services industry (e.g., trading, broker-dealer, asset management, investment management) is a strong plus.
  • Familiarity with cloud platforms like AWS and Snowflake for data storage and integration.
  • Excellent communication and collaboration skills, with the ability to work effectively with cross-functional teams.
  • A passion for learning and staying up-to-date on the latest advancements in data engineering technologies.

Powered by JazzHR

eCQwgVk2hh

Job Tags

Similar Jobs

Sarian, Inc.

Technical Writer Job at Sarian, Inc.

Must have skills* Demonstrable experience in Technical Writing skills Understands Software development methodologies and terminologies...  ...Data and Data visualization techniques Ability to produce content rich documentation Ability to understand data and recognize patterns... 

Get It - Healthcare

Mental Health Therapist - Remote | WFH Job at Get It - Healthcare

 ...the lives of individuals and families. Position Details: Remote Option Available (must be licensed in Michigan) Teletherapy Clinician...  ...their families Collaborate with a multidisciplinary team of healthcare professionals to coordinate comprehensive care Support... 

Allied Universal®

Security Guard - Hiring Immediately Job at Allied Universal®

 ...Job Description Allied Universal, North Americas leading security and facility services company, provides rewarding careers that...  .... Pay Rate $20.50 per hour Part Time - Evenings and Overnights - Flex Schedule and On Call. Ideal candidate would have 6+... 

Medical Search International

Physician Assistant, Dermatology Job at Medical Search International

*Urgent need for a Nurse practitioner /Physician Assistant specialized in Dermatology for the Portland, OR location**Schedule: Monday - Saturday. 5 Days per week - Would ask for flexibility with what day off during the week (Either Friday or Saturday). 8:30a-5:00p (3... 

The Kroger Company

Pace Dairy Foods 3rd shift Production & Sanitation Job at The Kroger Company

 ...timely manner; Organize and define tasks with minimal supervision; Participate in inspections and zone audits; Operate equipment associated with trash removal; Clean in a proper and timely manner to ensure safe, quality products...Hiring Immediately The Kroger Company