This was the week I wrote my supervisor a long honest email about everything wrong with my own tool, and got back a reply that reframed the entire project in about four sentences.
What went well
The rebuild worked. Coming out of last week's conversations I tore out the parts nobody had asked for and rebuilt around what the analysts actually said they needed.
Three things I'm genuinely happy with:
Every claim is now traceable to its original source in one click. Not a citation list at the bottom, not a footnote — click the claim, land on the sentence it came from. And the reasoning behind each suggestion can be followed end to end, so there's nothing you have to take on faith.
The data sources got fewer and better. Instead of scraping widely and hoping, the tool now pulls from four things: the UNHCR live API for refugee and asylum-seeker numbers in an area, the IOM API for internally displaced person counts by country, ACLED for violent conflict reporting from the last month, and GI-TOC's own past Libya reports. Four solid sources beat twenty shaky ones, and it makes the traceability promise actually keepable.
And the ranking of suggested interventions holds up. When I read down the list, the order makes logical sense, which was not true two weeks ago.
I also applied the framework to the Agadez case, as planned back in February. That was instructive in a way I didn't enjoy. My data sources and the report archive I'm grounded in are Libya-heavy, because that's where I narrowed the scope in Week 3. Point the tool at Agadez and it gets thinner and more confident at the same time, which is the worst possible combination. It doesn't know that it doesn't know.
The other thing that happened this week came out of a corridor conversation rather than the project plan. I found out the office has a retrieval problem of its own — years of reports, briefings and field material, and no good way to find the thing you half-remember from two years ago. This is exactly what the analysts described to me last week when I asked what took the longest. I volunteered to take a run at it. It's not in my project outline. I did it anyway, partly because it's a tractable problem and mostly because it's the first time this summer I've been able to give something back to the people who have been giving me their time.
What I got wrong, and what my supervisor said about it
I wrote to Alex with the honest list. The parts that bother me:
The FEE scoring — feasibility, effectiveness, ethics — feels arbitrary. The numbers come out and they look authoritative and they are essentially vibes with decimal places. It needs a real method underneath, something like multi-criteria decision analysis, rather than a model being asked to rate things on a scale.
The impact page is worse. It's supposed to project effects over five years. What it actually does is find the most similar case out of roughly eighteen GI-TOC reports and present that as precedent. It can't overlay or triangulate multiple sources, because to do that I'd have to know the weighting coefficients between sources and understand the triangulation method a human analyst uses in their head. I don't know either of those yet, and I can't automate a method I haven't learned.
And the summary of all of it: the whole output feels like a librarian rather than a seasoned analyst. It retrieves. It doesn't reason.
I also put my longer-term worries in writing. The amount of training data, context and examples the model has is nowhere near what you'd need for a genuinely informed judgement, and closing that gap is an enormous data-gathering problem in itself. And we're standing in the same alleyway as Palantir, Bloomberg and quantitative researchers who have spent decades modelling how one change ripples through an economic, social or political system. Competing on their terms is not realistic in six weeks or in six years. If this is going to be useful, it has to hyper-scope down to something narrow enough to actually solve.
The one bright spot I offered was that a tool like this might serve someone who has to make a high volume of decisions in a very short time, and I used local police handling minor cases as an example. Against a human analyst with a week and decades of context, I don't think it competes. Against no time at all, maybe it does.
Alex's reply did two things.
The first was to take apart the police example, and I'm glad he did, because I had written it as an aside without thinking it through. My own requirements document says this is a pre-intervention research tool, not an operational system, and that the Libyan police are an actor the model represents rather than a user it serves. Handing a fast, confident, thinly-grounded tool to an enforcement actor making rapid decisions is close to the exact failure mode the constraints exist to prevent. I wrote a line that contradicted my own Week 3 document and didn't notice, because it sounded like a promising market. That's worth sitting with. The pressure to find a use case is a real pressure, and it will bend your ethics quietly if you let it.
The second thing he did was to reframe the librarian complaint. I had presented "it's a librarian, not an analyst" as the central flaw. His view was roughly the opposite: the librarian is the product. The analysts don't need a machine that reasons for them — they have decades of context and they'd be right not to trust a model's judgement over their own. What they lose time to is retrieval, structuring, and noticing the thing in the archive they'd forgotten. Build the thing that's genuinely good at that, and stop trying to build a synthetic analyst that would be worse than the real ones sitting ten feet away.
So the hero use case narrowed to something I can actually describe in one sentence: a researcher opening an unfamiliar case, in their first day or two, needing to know fast what the archive already knows about the actors, the routes and what happened last time something similar was tried. Not prediction. Orientation.
It also means the side project I volunteered for and the main project are the same problem wearing different clothes, which I did not see until Alex spelled it out.
What could have been done differently
I should have sent that email a week earlier. Everything in it was true on Monday and I spent four days polishing the tool before admitting to the list. Some of that was wanting to bring solutions rather than problems, which sounds mature and functions as delay.
What I learned about myself when working with others
I've been treating criticism of my work as something to survive rather than something to collect. The email I dreaded sending produced the most valuable four sentences of the summer. There's a lesson there that I'll have to keep relearning.
I also noticed I'm drawn to the impressive-looking parts of a problem. Left alone, I optimise for what demos well. That instinct comes straight from pitching a startup, and it's actively harmful here.
What I learned about leadership
Two things.
Naming your own work's weaknesses to the person supervising you is not a confession, it's the mechanism by which the work gets better. Alex could not have redirected the project if I had sent him a summary of the good parts. The quality of the guidance I got was set by the quality of the honesty I offered.
And the pressure to justify a project will quietly reshape what you're willing to build. My police-use-case line was written by someone looking for a reason the tool matters, not by someone thinking about who ends up on the other side of the decision. Nobody made me write it. That's the point. The constraints I wrote in Week 3 did their job precisely because I'd written them down before there was any pressure on them.
What I want to focus on next
Write the second engineering document with the narrowed hero use case at the centre. Build an MVP focused on orientation and retrieval rather than prediction, and cut the impact page back to something honest about its own limits. Get the same analysts to try to break it. And get the office's retrieval problem to a state where it keeps being useful after I fly home.