How do you set complex projects up for success?

Original prepared answer

What does success mean?

• The project is delivered within the deadline

• Meets all functional and non-functional and quality goals

• The team has high morale after delivering it

Impact Assessment and PRD review (Goals and Margin dashboard)

Before we as a team decided to pick a project, we spent a lot of time understanding the project's impact.

So whenever a product or business comes to engineering with project requirements, my team and I try to analyze various aspects regarding its impact.

The most important parameters that we look for are

• Who is going to be users, are they internal or external?

• What revenue implications does it have

• What is the scalability opportunity? are eg. How many customers are going to adopt the product? Or how customers are already looking for it.

• Success metrics and how to measure it

Once the team is convinced that the project has significant impact and aligns with our North-star metric, we then decide to put it in our team's quarterly roadmap.

Our product team working on the core platform worked on interviewing internal traders we realized that they spend a lot of time investigating margin making opportunities for clients and ad-campaigns. It is a time consuming process and many times they miss-out making appropriate changes in the settings of the campaign that would result in missing out on maximum margin realization for Joveo. We are essentially leaving money on the table.

In terms of figuring out the impact , We then tried to figure out how much money we were talking about. Upon analyzing the data we realized that there are many instances where a client's goal is achieved before the budget is exhausted but we keep publishing those jobs.

Here we could make some margin by keeping the rest of the budget and not spending it.

We calculated that the total amount accrued for would be around 30K USD per month and it affects around 200 clients. The dollar no: would increase as we increase the no: of customers.

Also upon primary research we figured that every trader spends around 1 hour everyday to hunt for these situations manually and make amends. It counts to around 50 hours per week.

With this problem statement the product team worked on to create a product suite. Once they have created a PRD with all the details of

• User flows

• UI designs

• Actions

• Success metrics

• Success measurement mechanisms

Engineering team picks it up

High-level estimate and cross-team alignment

Once we have decided a project needs to be worked on. We do a quick high-level estimate of the total engineering day and try to map it to a set of engineers and also assign a single-point-of-contact. The spoc is usually someone senior who has experience in designing a system of similar scale and who has exp in delivering projects of similar scale. We also try to finalize how much effort is required from other teams like cross-engg teams, QA, UX, Fronten, legal etc.

• We do that to get bandwidth from all dependent teams and also so that they can put it in their roadmap and hold ownership for their part of the tasks

In this project we have few teams involved. FE and UX team for creating the UI, Notification team for push notification and Data Platform team which will be used for warehousing.

Stakeholder alignment

Once we have all this information, we set up an alignment meeting with various stakeholders like the product, sales, and marketing. In this meeting, we give out information like the scope of the project, the estimated time it might take, and the group of engineers who are going to be working on it.

• provide visibility of engineering line items and get buy-in from everyone

• allowing stakeholders to raise concerns regarding the usability or the impact of the product

• provide visibility of high-level timeline of delivery

Project planning; ERD creation; Design review

Once everyone is aligned on a high level, we work on a

• Detailed analysis of the functional and non-functional requirements.

• Break the project into phases by ruthlessly prioritizing sub-features of the proposed project

• Create high-level and low-level design and get it reviewed

• Bigger the project, more time spent here

• Emphasize on extendability of the project and make it future proof

• We break the project into the smallest possible unit of tasks.

• Identify every detail of cross-service requirements and pass the information on the requirements. And make the cross-team add it in their roadmap and get an ETA from them.

• Group tasks to identify milestones and their ETA (Contract for FE integration, APIs layer, service and repo layer, batch processor, event consumer, metrics and dashboard, load testing, ready for QA, ready for UAT)

• Assign tasks to engineers in the team and try to parallelize the independent components

• Once things are finalized, we publish an engineering review document

• Absolute scope of the project

• Design details

• Implementation details (service name, DB Tables, event structures, APIs)

• Detailed effort estimates and timeline using gantt chart

• Cost of building the project in terms of Infra

• SLAs

In this phase we happened to have a very important negotiation and long term decision. After we closed the discussion, we decided to split the project into 3 phases.

• The first phase will have the most valuable features and will be released only for internal traders. We would capture the adoption and plan the next phases (1.5 months)

• Second phase would have the good have features to increase adoption (2.5 Months)

• Third phase would have modified UI and more security features and notifications channels for external customers (4 Months)

We presented the technical design to the panelist. We decided to write it in Spring Webflux, use Snowflake as our data warehouse and leverage Notification service for push notification in our slack channels.

The design and timelines were approved.

Sprint Planning

• We create Jira tickets for every task and group them into epics (project notation)

• Each jira ticket has details of the work that needs to be done. The effort required in terms of story points; and ETA, and the owner of the task and links to dependent tasks if any

• The jira state workflow is controlled by our release manager software. When a developer merges their code, they need to put a jira no: to the git commit.

• And based on this jira no: our release manager identifies the jira ticket. The jira ticket closes only when the associated code is deployed in production.

• Assign subfeatures and tasks to individual engineers based on their bandwidth and availability

• After the tickets and epics are created, we align the tasks along the sprints of the quarter. Sprint is a two weeks period which we use to complete a unit of task.

• If there are cross-team items, we try to create tickets in their project as well and get them to add the ETA for those tasks as well.

Project Scrum calls

The project is driven by the engineering team that has the maximum work. And a SPOC is appointed from this team.

Once development has kicked in and has reached a certain point we start with alternate day scrum calls. We have alternate day calls. We also maintain a temporary slack channel, where the working group post scrum notes, updates, blockers or use it for escalations as well. All participating members like engg managers, directors and VPs are also present in this slack channel.

• this is done to resolve product doubts that various teams get during the course of the project

• This done to also get micro-prioritizations right

• Cross-team integration doubts

• Product Features

• service revamp integrations

• Discuss blockers

• In this call we get to know if the project is running on time or not. and if something seems off, we try to brainstorm on how to get things back to normal.

Pre QA exercises

Before we give out build to QA for testing, we have internal exercises to guarantee quality of our deliveries.

• Integrated bug bash

• Internal developers who did not work on this project are asked to break the system by doing random things.

• Load testing

• We evaluate the estimated traffic patterns and test our system on that scale

• Metrics and dashboards

• We identify the right metrics that needs to be kept handy for observing the health of the software

• EQR suite creation

• We have a EQR team who creates a end-to-end suit that runs in our deployment pipeline to validate integration correctness

QA Cycle

QA evaluates the product feature as a whole and not by each team's work.

• They look for overall sanity and correctness of the data visible in the feature

• They check if the actions are persisted

• UI related issues like if the button clickable or margins are symmetric on all browsers

• They raise bugs on jira and these tickets are triaged in the regular scrum calls that we have

Pre Release

This is done after QA cycle and before release

• Production readiness review

• In this we review the load testing results

• Metrics, logging, tracing

• Security

• Failure points

• Deviation of initial architecture

• UAT cycle

Release

• Usually, the back end is released very frequently as we keep developing.

• We closely monitor the order of release. We create a DAG of dependent components in our release sequence to avoid dependency failures

Post-release

• After a product is released, the product team publishes an announcement in the company about the rollout and plans the adoption

• The team focuses on observing the health dashboards and monitors the SLAs

• After few months of the release, the product team evaluates the success metric to see if the project was a success from business's standpoint

Retrospective

Engineering team does a retro exercise to look back and figure out areas of improvement and appreciate the wins.