🔍 a Search Engine for Yarn.social pods the Twtxt ecosystem https://search.twtxt.net/
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
 
 
 
 
 
 
James Mills 19127bf374
Update maxScrapers to 2 (from 50) to slow down scraping and reduce CPU and Disk I/O
8 months ago
..
pages Fix typos and switch to HTTPS links (#6) 9 months ago
session Remove session not found warning 9 months ago
static Remove more useless code borrowed from yarn's codebase 9 months ago
templates Fix paging and preserving sort 9 months ago
archive.go Refactor 9 months ago
base_task.go Avoid adding scrape/crawl tasks that already exist and are running 9 months ago
bitcask_store.go Fix a whole bunch of bugs I can't be botehred explaining 9 months ago
config.go Remove more useless code borrowed from yarn's codebase 9 months ago
context.go Add Dead Feeds to Stats page 9 months ago
crawl_task.go Update maxScrapers to 2 (from 50) to slow down scraping and reduce CPU and Disk I/O 8 months ago
crawler.go Avoid adding scrape/crawl tasks that already exist and are running 9 months ago
dispatcher.go Fix some concurrent bugs 9 months ago
dispatcher_test.go Initial Commit 1 year ago
email.go Remove more useless code borrowed from yarn's codebase 9 months ago
entry.go Improve crawler/scraper performance and add metadata about crawled feed urls 10 months ago
errors.go Initial Commit 1 year ago
func_task.go Initial Commit 1 year ago
handlers.go Add support for application/json content-negogation to search requests/responses 9 months ago
indexer.go Improve crawler/scraper performance and add metadata about crawled feed urls 10 months ago
init.go Initial Commit 1 year ago
jobs.go Add Twter to URL model and add job to fix null DiscoveredAt fields when Twter is populated 9 months ago
links.go Fix a whole bunch of bugs I can't be botehred explaining 9 months ago
models.go Add Twter to URL model and add job to fix null DiscoveredAt fields when Twter is populated 9 months ago
opensearch_handler.go Initial Commit 1 year ago
options.go Remove more useless code borrowed from yarn's codebase 9 months ago
paginator_adapters.go Initial Commit 1 year ago
results.go fix_template_loading (#5) 10 months ago
robots.go Initial Commit 1 year ago
router.go Migrate to Go1.16 embed (drop rice) 10 months ago
scrape_task.go Fix CI 9 months ago
scraper.go Add heuristic for feeds ending with a .txt extension other than twtxt that signifies the feed's cannonical name (e.g: 8ball.txt) 9 months ago
server.go Remove more useless code borrowed from yarn's codebase 9 months ago
session_store.go Renamed project and package to yarns (plural of yarn) 10 months ago
stats.go Refactored stats handling 9 months ago
store.go Fix a whole bunch of bugs I can't be botehred explaining 9 months ago
support_handlers.go Renamed project and package to yarns (plural of yarn) 10 months ago
task.go Avoid adding scrape/crawl tasks that already exist and are running 9 months ago
task_handlers.go Fix /tasks endpoint 9 months ago
templates.go Refactor crawler and scraper and remove a bunch of unused code 9 months ago
ttlcache.go Fix a bunch of data race bugs and deadlocks. Improve stats 9 months ago
types.go Add support for application/json content-negogation to search requests/responses 9 months ago
utils.go Refactor crawler and scraper and remove a bunch of unused code 9 months ago
worker.go Initial Commit 1 year ago