Useful tools and resources

This page describes a variety of tools and resources which may be useful when you want to employ NLP-based, quantitative, and/or computational research methods when researching governmental data.

Creating search engines

Sometimes it might be useful to create a custom search engine for your data. Some helpful tools or resources for this may be:

• awesome-selfhosted’s list of search engines
• or check out openbesluitvorming.nl as an example. openbesluitvorming.nl is a search portal by the Open State Foundation for searching through meeting documents of local democratic assemblies.

Creating & analyzing datasets

• DuckDB can be a useful tool to get to know your dataset and perform complex queries. Also check out awesome-duckdb, a list of DuckDB related resources and tools.
• Aleph is a tool primarily developed for (investigative) journalism, but may prove useful for an initial exploration of a (big) collection of PDF, Word and other documents.
• OpenRefine can help you refine and clean up your dataset.
• LawNotation is an open source annotation platform for analyzing the linguistic and legal characteristics of legal documents.

Working with API’s

• When working with a new API, it can be useful to interactively experiment with how the API works. API development tools such as Hoppscotch can be very valuable.

NLP toolkits

• spaCy (Python)
• NLTK (Python)
• Gensim (Python)

Last updated: 19-Dec-2024