🔍 a Search Engine for Yarn.social pods the Twtxt ecosystem https://search.twtxt.net/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
James Mills 19127bf374
Update maxScrapers to 2 (from 50) to slow down scraping and reduce CPU and Disk I/O
1 month ago
..
pages Fix typos and switch to HTTPS links (#6) 2 months ago
session Remove session not found warning 2 months ago
static Remove more useless code borrowed from yarn's codebase 2 months ago
templates Fix paging and preserving sort 2 months ago
archive.go Refactor 3 months ago
base_task.go Avoid adding scrape/crawl tasks that already exist and are running 2 months ago
bitcask_store.go Fix a whole bunch of bugs I can't be botehred explaining 2 months ago
config.go Remove more useless code borrowed from yarn's codebase 2 months ago
context.go Add Dead Feeds to Stats page 2 months ago
crawl_task.go Update maxScrapers to 2 (from 50) to slow down scraping and reduce CPU and Disk I/O 1 month ago
crawler.go Avoid adding scrape/crawl tasks that already exist and are running 2 months ago
dispatcher.go Fix some concurrent bugs 2 months ago
dispatcher_test.go Initial Commit 10 months ago
email.go Remove more useless code borrowed from yarn's codebase 2 months ago
entry.go Improve crawler/scraper performance and add metadata about crawled feed urls 3 months ago
errors.go Initial Commit 10 months ago
func_task.go Initial Commit 10 months ago
handlers.go Add support for application/json content-negogation to search requests/responses 2 months ago
indexer.go Improve crawler/scraper performance and add metadata about crawled feed urls 3 months ago
init.go Initial Commit 10 months ago
jobs.go Add Twter to URL model and add job to fix null DiscoveredAt fields when Twter is populated 2 months ago
links.go Fix a whole bunch of bugs I can't be botehred explaining 2 months ago
models.go Add Twter to URL model and add job to fix null DiscoveredAt fields when Twter is populated 2 months ago
opensearch_handler.go Initial Commit 10 months ago
options.go Remove more useless code borrowed from yarn's codebase 2 months ago
paginator_adapters.go Initial Commit 10 months ago
results.go fix_template_loading (#5) 3 months ago
robots.go Initial Commit 10 months ago
router.go Migrate to Go1.16 embed (drop rice) 3 months ago
scrape_task.go Fix CI 2 months ago
scraper.go Add heuristic for feeds ending with a .txt extension other than twtxt that signifies the feed's cannonical name (e.g: 8ball.txt) 2 months ago
server.go Remove more useless code borrowed from yarn's codebase 2 months ago
session_store.go Renamed project and package to yarns (plural of yarn) 3 months ago
stats.go Refactored stats handling 2 months ago
store.go Fix a whole bunch of bugs I can't be botehred explaining 2 months ago
support_handlers.go Renamed project and package to yarns (plural of yarn) 3 months ago
task.go Avoid adding scrape/crawl tasks that already exist and are running 2 months ago
task_handlers.go Fix /tasks endpoint 2 months ago
templates.go Refactor crawler and scraper and remove a bunch of unused code 2 months ago
ttlcache.go Fix a bunch of data race bugs and deadlocks. Improve stats 3 months ago
types.go Add support for application/json content-negogation to search requests/responses 2 months ago
utils.go Refactor crawler and scraper and remove a bunch of unused code 2 months ago
worker.go Initial Commit 10 months ago