MiniSearch Example – Daily Drop Companion

This is a companion piece to Drop #348 to show how to use MiniSearch.

The source for the live, standalone versions of what we’ll walkthrough here, today, is over at GitLab. What’s here will follow closely to what’s in bare-bones.html. It’s also on GitHub but I’m only doing that because folks still “live” there. You will 100% want to keep the bare-bones.html source up as you follow along. You may also want to keep the MiniSearch documentation site up as well.

Make sure to poke at the live versions of the bare-bones, standalone example and Vite-built, slightly more styled example before continuing.

KEV MiniSearch Set Up

There are two basic components to this “app”. The first is an input text box we’ll be monitoring for changes. The second is a results list that dynamically changes depending on what you type in the input text box (no “submit” required).

Inside the body tag, there is a single, custom Lit element:

<kev-search />

It takes care of displaying the search box, monitoring the search box, and displaying the results.

We’re using both MiniSearch and Lit in this minimal example, so we need to tell the browser that (the rest of this companion is assuming we’re in that solo <script> tag):

import MiniSearch from 'https://cdn.jsdelivr.net/npm/minisearch@6.1.0/+esm'
import { LitElement, css, html } from 'https://cdn.jsdelivr.net/npm/lit@2.8.0/+esm'

If you are not a fan of shunting visitors telemetry to giant CDNs, you can use my esmdl Golang utility to mirror them locally (you’ll need to import all of what that utility downloads).

Next, we need the data we’re going to search through. I mirror KEV daily, and have a generous CORS policy on one directory of my main domain’s website, so we can use that:

// Fetch and deserialize the JSON
const kev = await (await fetch("https://rud.is/data/kev.json")).json()

// MiniSearch needs a unique identifier for each record 
// named `id` so we'll just reuse the CVE id since that
// will always be unique.
const vulns = kev.vulnerabilities.map(d => {
  d.id = d.cveID
  return (d)
})

// When returning search results, MiniSearch does so
// by the `id` (above). All we do here is build a
// disctionary so we can quickly get to the record 
// contents.
const kevByCVE = vulns.reduce((byCVE, entry) => {
  byCVE[ entry.id ] = entry
  return byCVE
}, {})

Now, we need to setup our MiniSearch search engine by populating it with the KEV data (see the constructor docs for what the parameters mean and feel empowered to tweak them):

const kevSearch = new MiniSearch({
  fields: [ 'cveID', 'product', 'shortDescription', 'vendorProject', 'vulnerabilityName' ],
  storeFields: [ 'cveID', 'shortDescription', 'dateAdded' ]
})

kevSearch.addAll(vulns)

MiniSearch has a bonkers number of ways you can configure its behavior. Our example needs are minimal:

const searchOptions = {
  fuzzy: 0.1,
  prefix: true,
  fields: [ 'cveID', 'product', 'shortDescription', 'vendorProject', 'vulnerabilityName' ],
  combineWith: 'OR',
  filter: null
}

fuzzy: Controls whether to perform fuzzy search. It can be a simple boolean, or a number, or a function. If a number between 0 and 1 is given, fuzzy search is performed within a maximum edit distance corresponding to that fraction of the term length, approximated to the nearest integer.
prefix: Controls whether to perform prefix search. It can be a simple boolean, or a function.
fields: Names of the fields to search in. If omitted, all fields are searched.
combineWith: The operand to combine partial results for each term.
filter: Function used to filter search results.

The remainder of the JavaScript code is for the Lit element. There are two properties managed by the element:

searchText: a String object that makes our input box reactive
results: an array of search results from MiniSearch that will change as we type. Each change causes a fresh call to the render() function.

I dislike it when web apps with clear, default input fields aren’t given the focus when a page installs. We’ll be kind to our users and have the Lit element do just that after it’s set up everything:

firstUpdated() {
  this.shadowRoot.querySelector('#kev').focus();
}

We make our input box reactive with this bit in render():

<input placeholder="enter search terms…" 
       type="text" 
       name="kev-input" 
       .value=${this.searchText}
       @input=${this._getSearchResults}
       id="kev"/>

.value binds the input box contents to our searchText property
@input ensures we make a new call to _getSearchResults() each time the field changs; let’s see what that looks like:

_getSearchResults(e) {
  e.preventDefault(); // the processing of this event stops here
  this.results = kevSearch.search(e.target.value, searchOptions).map(({ id }) => kevByCVE[ id ])
}

We feed the current value of the input text and the searchOptions to the call to search() and then use the dictionary we made to retrieve the entries returned in the resultset.

To display the results, we iterate through the results, making a bunch of custom HTML tag elements that are essentially just semantically named <div>s. We pull in the CVE, vulnerability name, and the short description:

${this.results.map(r => html`
<entry>
  <cve>
    <a href="https://nvd.nist.gov/vuln/detail/${r.cveID}" target="nvd">${r.cveID}</a>
  </cve>
  <vuln>${r.vulnerabilityName}</vuln>
  <descr>${r.shortDescription}</descr>
</entry>

Your Turn

Here are some ideas to help experiment more with our KEV MiniSearch app:

KEV could eventually get yuge; use some “loading…” indicator idiom (which will also make this kinder to folks on slow connections, now)
tweak the searchOptions manually
look at the MiniSearch author’s example code to see how to let users modify the searchOptions
the app starts searching immediately upon the first keystroke; that’s sub-optimal. Modify the code so that there needs to be a minimum threshold character count before performing the search.
the MiniSearch author’s code has some other search nice-to-haves such as a “clear” button, and a pop-up with the best suggestions. Consider adding those to this example.
perhaps consider reimagining the author’s example application by using Lit elements.
if you end up trying to work with the larger example KEV MiniSearch application code, consider following some of the guidance in Thursday’s Drop and re-style it according some some deliberate design system choices you make on your own.

FIN

If you run into any issues, don’t hestiate to ping me wherev we both hang. ☮️