0.6.1

This is bugfix release only: the templates for the web interface were not included in the release artifacts.

0.6.0

This release improves the API significantly by allowing for a clearer API between the corpus and the index, and also with respect to customised rendering of values for fields in different contexts. This will be further exploited in the future to allow more customised rendering of values in the web UI, and enable enhanced visualisations such as trendlines in the future.

• Improves UI by allowing rendering of matching snippets (concordance lines or keyword in context) for a query, along with the matching features and the document itself.
• Improves clustering algorithm by making acceptance of moves depend on the ratio of cluster sizes: this prevents clusters from being emptied out too quickly and limits the accumulation of features into a single large cluster.

• Significant improvements in the handling of stackexchange data: you can now work with the .7z archive files directly on the command line. You will need to install extra dependencies first though:

  # Install optional dependencies along with the upgraded library
  
  $ pip install --upgrade hyperreal[stackexchange]
  
  # assumes a stackexchange archive file has been downloaded to the current directory.
  # Will replace the contents of expatriates stackexchange into the expat.db
  
  $ hyperreal stackexchange-corpus replace-sites expatriates.stackexchange.com.7z expat.db
  

Note: because of the change in the corpus API and the migration API, it will likely be simpler to recreate your indexes from scratch against the new API rather than try to migrate in place. If you do want to migrate your data in place you will need to make sure your index database has been migrated first in version 0.5.0.

0.5.0

Further refinements and simplifications of the clustering process - it is now possible to control the grouping size in the hierarchical clustering improvement.

This release also migrates the packaging infrastructure to fully use pyproject.toml.

0.4.1

• Bugfix for an edge case in cluster refinement. #64
• Very rough initial implementation to allow searching for a single value in a single field - this currently only supports exact matches and has no error handling.

0.4.0

Lots of new features in this release:

• support for network export of cluster-cluster cooccurrence via networkx (see the new export subcommand on the CLI, or the Index.create_cluster_cooccurrence_graph method.
• support for structured sampling using the model as a guide for sampling documents
• interface for corpus implementations to support exporting tabular renderings of documents
• refactoring of the clustering method so more of the pieces can be used on their own - this will be used for further functionality in the future
• reworking of the web UI render pathway, so the top of the page is calculated first and streamed as soon as it's ready - this enables better on interactivity on very large datasets
• more sophisticated tokenisation of stackexchange data: codeblocks are now pulled out and indexed separately.
• initial examples for two large scale document collections: stackoverflow and Australian Federal Hansard

Experimental initial features:

• initial experimental indexing of word positions to enable phrase and proximity queries - this is likely to change a lot in future releases
• initial support for rendering of concordances (keywords in context)
• initial support for intersecting a set of queries with a given field - this will be used in the future to enable first class support for temporal queries

0.3.1

This is a small utility release:

• make available a bugfix that prevents zombie processes from automatically created process pools
• factors out some functionality from the feature clustering algorithm into their own methods to allow more flexibility in future.

0.3.0

This release includes a number of tweaks and improvements to support some useful workflows for editing and manipulating models after the initial computational feature clustering. It also updates the clustering algorithm slightly so it should converge faster and make more interesting clusterings when expanding the number of clusters.

The main component of this release is a new and improved approach to the clustering algorithm. This results in improvements in the objective score, and enables dynamically reducing the number of clusters in the model in addition to increasing. This might be used in the future to allow new complications and changes for the clustering algorithm too.

• Make PlainTextSqliteCorpus take a custom tokeniser function by @SamHames in #53
• Skip Twitter related tests in CI due to Twitter API changes by @SamHames in #54
• Refine the clustering algorithm by @SamHames in #55

0.1.1 - Fix packaging related issues

This release is a bugfix to the distributed package, the wheel pushed to pypi wasn't usable because it was missing key things.

0.1.0

This is the initial release of the Hyperreal package for interpretive topic modelling.