This week in “what we have done”: lots of debugging, making everything too complicated (but working), a rough plan for the next two months.
- Finished some big steps integrating our crawlers with the indexer. we can actually collect the repo data now! :)
- Added some additional setup to the repos, like bumpversion and automated tests
- Wrote some additional logic to export the data automatically for finished hosters
doing / motivations and challenges / time schedule
We are heading to finish a first working version of the indexer, but there is still a lot to do. Our state handling is a bit overcomplicated in some parts, but since it works, it has to stay like that until we finished integrating everything with the actual search. The next big point when we sorted out some last bugs and everything running a bit smoother, is a long running crawler- and indexer setup, and then find out how we unify the different results, to throw them in our search backend.
In terms of our time schedule this means, that we hope to have “everything” working together in about four weeks - and then we have another four to document all the new parts, how everything works together, and hopefully clean some parts of the code up. :)