Yahoo
Software Engineer I
Dublin, Ireland · February 2024 — Present
- 01Built a real-time system that classifies web pages into ad-targeting segments, using Kafka (MSK), Spark Structured Streaming and OpenSearch. It feeds pipelines that process events each day.
- 02Found the cause of a slow search cluster: only 2 of nodes did the work. Fixed it and cut batch time from to 5 minutes. Throughput went from to records per second.
- 03Used an LLM (Claude on AWS Bedrock) to write keyword rules for page classification. Tested on URLs, it halved search queries per page at recall.
- 04Co-designed the new version of a scoring pipeline that handles events a day across segments. Compute cost fell by net, and scores now update several times each day instead of once.
