Skip to content

Data Science

How a simple data science habit—a tiny eval set and accuracy@k—helped engineers and PMs turn "this looks okay" into a defensible decision in one hour.

The problem

Our product lets users search a database of emission factor names. Think entries like "Copper products", "Crude petroleum and services related to crude oil extraction, excluding surveying", or "Aluminium and aluminium products". Users type short, messy queries: typos ("cupper"), partial names ("crude petroleum"), regional spellings ("aluminum" vs "aluminium").

We were migrating search off a dedicated search engine and onto plain Postgres. The question on the table:

Can an out-of-the-box Postgres extension (pg_trgm) give us good enough search, or do we need something heavier: embeddings, a vector database, a Hugging Face model, a custom bag-of-words ranker?

With AI, a product manager or engineer can quickly implement several PostgreSQL search options, even without deep search expertise. The same tools can also help simulate realistic user queries and build a small evaluation set in under an hour. That makes it possible to compare the options with evidence instead of stopping at "this looks okay."

When One Company Appears Under Ten Different Names

Company names often look like clean identifiers in a database.

In reality, they rarely are.

In one 811 organization's historical ticket data, contractor names were entered as free-form text. The same company could therefore appear under several variations:

  • GA Power
  • GA Power Co.
  • GA Power Company

Other differences came from punctuation, abbreviations, spelling mistakes, legal suffixes, or inconsistent spacing.

To a person, these clearly look like the same organization. To a database, they are separate entities.

How a Simple Data Analysis Reduced Unnecessary Audit Work

Not every useful data project needs a complex model.

Sometimes, a simple analysis can reveal that an existing business rule is creating unnecessary work.

A utility notification organization had a policy of auditing every ticket submitted by a new user for their first 30 days.

The reasoning was straightforward: new users were expected to make more mistakes while learning the system.

But the 30-day threshold was based on a heuristic rather than data.

Since audit capacity was limited, we wanted to answer a simple question:

Do new users really need to be audited for the full 30 days?