Applied ML evaluation

The question behind the model

Observational data contains many reasons for an apparent relationship. The measurement process, differences between subjects, and the conditions around an observation can all affect the result. A useful analysis needs to examine those alternatives before turning a pattern into a product claim.

I contributed to applied ML evaluation and scoring using double machine learning. My work included cross-fitted models, candidate feature evaluation, and investigation of whether an apparent signal reflected the behavior we intended to measure.

My contribution

I worked with XGBoost and LightGBM, evaluated features, and examined how conclusions changed under different modeling and validation choices. Cross-fitting helped separate model fitting from estimation, while sensitivity checks made it possible to see which assumptions mattered to a conclusion.

The data work was just as important as the modeling. I investigated measurement artifacts and differences in how observations were grouped. These checks helped distinguish a plausible numerical result from one supported by the underlying data.

I also worked on how the analysis connected to application behavior. An estimate needs a clear definition and enough provenance to be interpreted consistently. The engineering responsibility continues after model fitting: the meaning of the output must survive its use elsewhere.

What I take from it

This work made me more careful about the distance between an estimate and a demonstrated outcome. Observational analysis can inform a decision, but a modeled opportunity does not establish that an intervention delivered a benefit.

That habit carries into my infrastructure work. I ask what a test actually measured, which assumptions its conclusion depends on, and whether another person can reproduce the result. I value analyses that preserve uncertainty clearly enough for an engineer or domain expert to make the next decision with confidence in what is known.