Log

• Initial work on searchAfter pagination 💬 by Caio 7 years ago

  This unfortunately can't be done as-is because of the sort()
  functionality: in order to correctly recreate the reference
  point for the subsequent query we need to also serialize the
  `fields` field from the FieldDoc result and that's an
  Object[] (in our case it's always floats, but the api doesn't
  guarantee this).
  

• Merge branch 'pagination' by Caio 7 years ago
• Add basic pagination support 💬 by Caio 7 years ago

  This patch implements pagination by means of holding all items
  in the result until the desired offset is reached. This results
  in a functionality that works well for the first pages but
  grows progressivelly more expensive the further we are from
  the top results.
  
  A better theoretical approach would be to hold a reference to
  a previous bottom item (as in: its Id and Score (and more if
  sharding)) and use it as the threshold so that instead of keeping
  every single ranked item in memory we can discard everything
  that scores higher than it (meaning: memory usage becomes a
  function of the page size, not page size + offset). Lucene
  provides a functionality similar to this via searchAfter
  but it isn't super trivial to use because the sorting API is
  untyped so we need to be more careful creating the reference
  point instance.
  
  This patchs implementation works ok, especially with <1M documents
  to be looking at, and it's easier to roll since it's just a one
  way communication channel (front-end just needs to specify the
  page number- searchAfter requires a previous search result to
  know the pagination parameters). I think I'll keep it.
  

• Disable the whitelabel error page by Caio 7 years ago
• Merge branch 'html' by Caio 7 years ago
• Bump jackson to 2.9.8 by Caio 7 years ago
• Upgrade to lucene 7.6.0 by Caio 7 years ago
• Wrap search calls with a circuit breaker 💬 by Caio 7 years ago

  Make sure we give some time for the service to recover if we enter
  a situation where the error rate goes too high.
  
  This patch introduces a circuit breaker for the calls that go
  through the parallel scheduler so that we stop accepting queries for
  a while (60s by default) if the error rate gets too high (50% by
  default).
  
  A health check (under /actuator/health/circuibreaker) has been added
  and metrics are exposed via /actuator/prometheus and
  /actuator/metrics/resilience4j.*
  

• Render responses in html 💬 by Caio 7 years ago

  Turns out playing with more than one box gets expensive very
  quickly. Who knew!
  
  This patch introduces a dummy / endpoint, where the index page will
  live; And a /search endpoint that acts as the previous one, but now
  rendering an html result.
  
  Using [rocker][1] as the template engine leads to some coupling as
  I'll always either return something that is/contains
  a RockerTemplate or is the output of it - I don't think it fits the
  current model spring uses to decouple controllers (I can make it
  slightly less coupled though - will do it in the future), but rocker
  is such a neat little project that I'll try to keep it :-)
  
  [1]: https://github.com/fizzed/rocker
  

• Default maxFacets to zero by Caio 7 years ago
• Add support for setting number of facets via &nf 💬 by Caio 7 years ago

  Collecting facets is the most expensive part of the query right now,
  so I'm thinking of disabling most of it. At least for a scenario
  where a term will hit too many docs.
  

• Searcher.search only wraps IOException 💬 by Caio 7 years ago

  Don't want to wrap every exception otherwise I risk ignoring
  unexpected things when eyeballing because "ah it is just a
  SearcherException again".
  

• Get rid of the Environment static helper 💬 by Caio 7 years ago

  I like my library dependency graph with little to no cycles.
  

• Parameter parsing only throws SearchParameterException by Caio 7 years ago
• Move all parameter parsing logic into a new class 💬 by Caio 7 years ago

  This way the hierarchy is less messy; Business logic of the web layer
  was sneaking into the core functionality.
  

• Add tests for the Controller's SearchQuery.Builder 💬 by Caio 7 years ago

  Which in turn detected a bug I hadn't caught yet: missing
  breaks in the parameter handling switch :-)
  

• Add support for filtering by number of ingredients by Caio 7 years ago
• Introduce RangedSpec.fromString() 💬 by Caio 7 years ago

  This patch makes RangedSpec capable of parsing basic range
  input with the following rules:
  
  * If the input doesn't contain "," it's treated as a range
    starting from zero and ending on Integer.parseInt(input)
  
  * If it contains "," then it should be in the format
    "numberA,numberB"
  
  I'm not sure I like this logic being here, but hey.
  

• Check SearchQuery.Builder.sort(String) functionality by Caio 7 years ago
• Leave validations to the builder only 💬 by Caio 7 years ago

  There's nothing complex going on here, no point in validating
  parameters twice in the chain.
  

• Merge branch 'webflux' 💬 by Caio 7 years ago

  Use Spring WebFlux instead of WebMVC
  

• Add support for timing out requests 💬 by Caio 7 years ago

  This patch sets up the `cerberus.search.timeout` configuration
  which govens how long we'll wait for a search query to complete.
  
  If the countdown expires before the search completes we'll
  still get a valid FailureResponse, with the error flag set to
  "TIMEOUT_ERROR".
  

• Use webflux instead of webmvc 💬 by Caio 7 years ago

  Reactor pattern + separate executor for lucene calls is a more
  reasonable concurrency model IMO.
  

• Use Path.of instead of Paths.get by Caio 7 years ago
• Merge branch 'boot' 💬 by Caio 7 years ago

  Integrate the `boot` branch into mainline. We're moving forward with
  this now.
  

• Drop withIngredients/withoutIngredients 💬 by Caio 7 years ago

  Not being used right now. If I reimplement this it will be through
  parsing the fulltext field, i.e.: querying for `keto ingredient:egg`
  would search for `keto` in the whole but `egg` would only be searched
  for in the ingredient field.
  

• Move searcher setup to a configuration component by Caio 7 years ago
• Better formatting by Caio 7 years ago
• Expose indexWriter.forceMerge(1) as mergeSegments() by Caio 7 years ago
• Move assertion properties to Util by Caio 7 years ago