2021-08-23 KW33

HubGrep works!

Okay, it’s been working for a while now to be honest, but behind the scenes we spent a lot of effort on ironing out wrinkles with our crawling cycle and tuning HubGrep to be more efficient with search responses and index updating. And not to mention that for a week or so we have a complete dataset for all our current hosters! At least that’s what we’re telling ourselves. In terms of progress, this mostly means we now have a full GitHub collection, finally.

So while we still want to improve the speed at which we can update our index, most important for us right now is to tweak search results to be better weighted. We also need to go over our documentation again since there have been some changes that need to be reflected.

done

• Shot a tiny talking-heads clip and wrote a blog post in preparation for Prototype Fund demo-week.
• Swapped out Sphinx for Manticore (which is based on Sphinx).
• Refactored and fixed some end-of-cycle automation for updating our search index.
• Upgraded our servers - we ran out of both RAM and disc space.

doing

• revisiting documentation
• tweaking search results
• re-tweaking search results