Big Data Architect

---

Job description

Big Data Architect

Location : San Clara bay Area CA OR Iselin NJ (Remote)

Duration : Full-Time

Data Architecture (Big Data, Hadoop, Realtime Streaming) - Sr. Enterprise Big Data Engineee/Architect

Will be responsible to design, develop, integrate, and maintain Enterprise level Big Data Systems with both batch and streaming datasets. Should have a very strong understanding of MPP databases, Shared Nothing, Shared-disk, and other Modern Big Data tech stack like Spark, Hadoop, AWS Redshift, Confluent Kafka, AWS Kinesis, and other Streaming technologies. This individual is expected to design Big Data systems based on Industry best practices and architectural guidance of Big Data systems, with understanding of integration with other data sources and tools like Markit, Business objects, Informatica, MS SQL, PL/SQL, etc. Enhance/Maintain/support existing applications. Collaborate with other developers in designing.

Must Skill:

• Partner with business leadership to identify problems, and opportunities for technology innovation with a focus on Big Data implementations.
• Help establish a clear, consistent technology vision through collaboration, influence, and enablement.
• Research, recommend, design and develop Big Data systems and with a sound understanding of the Big Data application architecture and Integration.
• Extensivepast hands on experience with MPP databases, Shared Disk, and Shared Nothing databases.
• Extensive experience and knowledge of Data streaming tools like Confluent Kafka (with Knowledge of kSQL), and any one of the other Streamining data ETL processing tools like Flink, Spark Stream API, Storm, Streamsets, Flume, Samza, Filtering, Windowing, Joins of streaming data.
• Strongly Understand ESP (Event Stream Processing) and Complex Event Processing (CEP) and Event Driven Architecture.
• Extreme understanding of CDC tools like Debezium, GoldenGate, Attunity/Qlik…
• Stay informed on initiatives across the industry and the enterprise to help product owners and IT leadership effectively prioritize.
• Tailor communication and convey complex ideas in a concise manner to a diverse set of audiences and venues.
• Identify and assess the organizational impact of enterprise architecture and standards, including change in skills, processes and structures with an emphasis on the DataWarehouse.
• Must be an expert in SQL and one of the ETL tools such as Informatica
• Extensive experience in SQL query tuning
• Creating robust and extensible data pipelines for production systems
• Creating secure, performant, and well-modeled data stores
• Must be fluent in any one of the scripting languages such as Python
• Experience working in an onsite client technical consulting environment preferred.
• Source code version control management using tools like Git/GitHub
• Experiences working within Agile Frameworks, such as Scrum or Kanban
• Excellent communication skills to be able to interact directly with non-technical client stakeholders and act in a business-to-technical translation role.

Preferred Skills:

• Knowledge and use of DataOps practices preferred
• Experience using NoSQL databases (e.g., AWS DynamoDB, MongoDB, Neo4j)
• Knowledge of Orchestration tools like Airflow.