Radical Translations Project Documentation¶
Table of Contents:
Radical Translations¶
TODO: Add Radical Translations description…
Settings¶
See detailed cookiecutter-django settings documentation.
Development¶
Local with Docker¶
See detailed cookiecutter-django development with Docker documentation.
Local without Docker¶
See detailed cookiecutter-django local development documentation.
Basic Commands¶
Setting Up Your Users¶
To create a normal user account, just go to Sign Up and fill out the form. Once you submit it, you’ll see a “Verify Your E-mail Address” page. Go to your console to see a simulated email verification message. Copy the link into your browser. Now the user’s email should be verified and ready to go.
To create an superuser account, use this command:
$ python manage.py createsuperuser
For convenience, you can keep your normal user logged in on Chrome and your superuser logged in on Firefox (or similar), so that you can see how the site behaves for both kinds of users.
Type checks¶
Running type checks with mypy:
$ mypy radical_translations
Test coverage¶
To run the tests, check your test coverage, and generate an HTML coverage report:
$ coverage run -m pytest
$ coverage html
$ open htmlcov/index.html
Running tests with py.test¶
$ pytest
Live reloading and Sass CSS compilation¶
Moved to Live reloading and SASS compilation.
Deployment¶
The following details how to deploy this application.
Docker¶
See detailed cookiecutter-django Docker documentation.
Custom Bootstrap Compilation¶
The generated CSS is set up with automatic Bootstrap recompilation with
variables of your choice.
Bootstrap v4 is installed using npm and customised by tweaking your variables
in static/sass/custom_bootstrap_vars
.
You can find a list of available variables in the bootstrap source, or get explanations on them in the Bootstrap docs.
Bootstrap’s javascript as well as its dependencies is concatenated into a
single file: static/js/vendors.js
.
Technical Overview¶
Team¶
Organisation: King's Digital Lab
Site: https://kdl.kcl.ac.uk
Email: kdl-info [at] kcl.ac.uk
Twitter: @kingsdigitallab
GitHub: https://github.com/kingsdigitallab
Location: London WC2B 5LE, United Kingdom
Research Software Analyst: Arianna Ciula
Site: https://www.kdl.kcl.ac.uk/who-we-are/arianna-ciula/
Research Software UI/UX Designer: Ginestra Ferraro
Site: https://www.kdl.kcl.ac.uk/who-we-are/ginestra-ferraro/
Research Software UI/UX Designer: Tiffany Ong
Site: https://www.kdl.kcl.ac.uk/who-we-are/tiffany-ong/
Research Software Engineer: Miguel Vieira
Site: https://www.kdl.kcl.ac.uk/who-we-are/miguel-vieira/
Technologies and Processes¶
Development¶
For more information see development and development with docker.
Data model¶
The Radical Translations project data model is based on BIBFRAME, for Resources (Works, Instances, Items are flattened under one object type) and Events, and based on FOAF for Agents (Persons, Organisations).

Django models¶
The data model graph was generated with the django-extensions graph_models command:
$ fab django "graph_models -X TimeStampedModel,PolymorphicModel -o models.png agents core events utils"
The following table shows the names of the fields in the data model and how they are being labelled for the search and detail views.
Model |
Field |
Search |
Detail |
---|---|---|---|
Resource |
title |
||
title_variant |
|||
subjects |
subject; form/genre |
topics; form/genre |
|
date |
Year slider |
||
edition_enumeration |
|||
summary |
summary |
||
held_by |
held_by |
||
electronic_locator |
URL |
||
notes |
notes |
||
classification |
printing and publishing status; paratext forms; paratext functions; translation status |
printing and publishing status; paratext forms; paratext functions; translation status |
|
contribution |
contributor; contributor role |
contributions |
|
language |
language |
||
place |
publication country; publication place |
||
relationship |
related resources |
||
relationship.relationship_type |
status |
||
Agent |
name |
||
radical |
radical |
||
based_near |
main place |
main |
|
page |
|||
roles |
role |
||
sources |
|||
notes |
notes |
||
Organisation |
members |
members |
|
Person |
given_name |
||
family_name |
|||
gender |
gender |
||
noble |
noble |
||
main_places |
other places |
other |
|
date_birth |
year slider |
||
place_birth |
|||
date_death |
year slider |
||
place_death |
|||
languages |
language |
||
knows |
knows |
||
member of |
member of |
||
Event |
title |
||
date |
year slider |
||
place |
country |
||
related_to |
related resources |
||
classification |
classification |
Conceptual model frameworks:¶
Controlled terms¶
The vocabularies, data points and taxonomies, used by Radical Translations data model:
FAST used for genres (subjects): this is part of the Library of Congress Linked Open Data resources mapped as FAST topics/forms
Additional terms for subjects (with respect to types of publications and genres) use terms from the RBMS controlled vocabularies (in particular the RBMS printing and publishing and RBMS genre vocabularies)
Wikidata for the professions (roles) of persons e.g. authors and translators as well as types of organisations e.g. political parties vs publishers; Wikidata terms are also used to express editorial classification e.g. of dates (e.g. inferred), events (typology) and relationships (e.g. uncertain attribution)
VIAF to identify some of the agents (persons) who are well known translators/authors
GeoNames for geocoded placenames
Extended Date/Time Format (EDTF) Specification to express dates
In one case controlled terms are based on project-specific terms; this is the case for ‘classification scheme edition’ for the Resources objects (in particular paratexts) with values partially adapted from Kathryn Batchelor, Translation and Paratexts (Roudledge 2018); Amy Nottingham-Martin,”Thresholds of Transmedia Storytelling” in Examining Paratextual Theory and Its Applications in Digital Culture, ed. Nadine Desrochers and Daniel Apollon, (IGI Global 2014), 287-307.
Workflows¶

Data workflow¶
Architecture¶

Local Docker Stack¶

Production Docker Stack¶
The graphs were generated by the docker-compose-viz tool:
$ docker run --rm -it --name dcv -v $(pwd):/input pmsipilot/docker-compose-viz render -m image local.yml
Design process¶
The design process followed to develop and deliver the Radical Translations project is derived from the double diamond process.
It consists of four phases with often multiple iterations happening within each phase.
Phase 1 – Discover¶
During this phase we discussed the project details and objectives with the research team. Understanding the data, contributing to the definition of the data model, defining and surveying potential audiences. Design workshops were also held to explore options and expectations at different points in the project and plan further design iterations.
Phase 2 – Define¶
The outcomes (e.g., reports) were aligned with the project requirements and tasks to start development were created (iteratively, following increments).
Phase 3 – Develop¶
Different solutions were offered (e.g., static and interactive mock-ups) to realise the vision of the project, component by component.
Phase 4 – Deploy¶
Usability testing and final implementation took place. Further adjustments happened in this phase to accommodate reactions from the wider audience interacting with the digital platform.
Design Workshops and User Research¶
Purpose: align high level requirements to use cases to drive data model definition and development (use cases lead to identification of usability testing units later in the evolutionary development cycle).
Mapping the data¶

Draft domain model¶
User types, scenarios and cases¶
Identify audiences and user types: anonymous survey sent to identified groups of interest and data collected over a 1-month period.
A report of the survey outcome was shared within the team.

A screenshot of the survey partial results¶
KDL and the research team collaboratively defined use cases and user scenarios.
The scenarios mainly focused on the search and retrieval of information in the search and browse section of the site (database).
Research¶
At this point KDL started work on the look and feel. Iconographic research, colour palette and imagery were collected to study and drive the creation of a design style that matched and reinforced the message and information the content is conveying.
Mock-ups¶
Next rounds of mock-ups were created. Following the mobile first approach, the focus was on prioritising content and interaction, rather than style.

First round of mock-ups¶
Content guidelines were also provided to support the research team generate accessible, and web suitable, content.
Adjustments were made based on comments and feedback.

Second round of mock-ups¶
The same approach was applied to specific components of the site, like the search and timeline visualisation.

Design iterations over the timeline component¶
More information on the process to create the timeline visualisation can be found in the paper “Small Data and Process in Data Visualization: The Radical Translations Case Study” published by the research team and available at: https://arxiv.org/ftp/arxiv/papers/2110/2110.09349.pdf.
Qualitative usability testing¶
Usability testing focused on the Database section of the site.
A set of tasks was created to test specific areas of the digital platform:
Top navigation
Secondary navigation
Search and Browse (Agents and Resources)
Timeline (Interactive visualisation)
Five testers were selected and interviews took place both in person and remotely. Findings where then shared with the team and further improvements were implemented based on the testing outcomes.
Changelog¶
All notable changes to this project will be documented in this file.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
[1.8.0] - 2023-07-28¶
Fixed¶
Accessibility issues.
[1.7.0] - 2023-03-24¶
Added¶
Mardown editing support for notes.
Design technical documentation.
ISBN to the footer.
Fixed¶
Issue with https urls not being returned by the API for pagination.
[1.6.0] - 2022-05-20¶
Added¶
Filter for anonymous Agents.
Wagtail search page type.
Inputs to the date slider.
Changed¶
Agent display name in the search results to the format family_name, given_name.
Scroll to results after interacting with filters.
[1.5.1] - 2022-02-22¶
Fixed¶
All Person agents being marked as private.
[1.5.0] - 2022-02-08¶
Added¶
Option to duplicate Resource records.
Authors of source text to other editions.
Fixed¶
Protocol headers not reaching Django/DRF.
[1.4.4] - 2022-01-20¶
Fixed¶
Ensure all resources are plotted on the timeline.
[1.4.3] - 2022-01-05¶
Fixed¶
Remove other edition relationships from source texts (it was added as a test).
[1.4.2] - 2021-12-20¶
Fixed¶
Style issues in the timeline.
[1.4.1] - 2021-12-20¶
Fixed¶
Read the docs build issue.
[1.4.0] - 2021-12-20¶
Added¶
Filter for private resources in the admin.
Style to preserve line breaks in notes.
Changed¶
Don’t mark as private Organisations that have members.
Removed¶
Cloudflare analytics integration.
Phrygian cap from topics.
Fixed¶
Roles not displaying for agents.
Duplicate contribution roles.
[1.3.1] - 2021-11-18¶
Added¶
Sidebar to display other pages in context.
Changed¶
Sort future events in ascending order, past events in descending order.
Fixed¶
Minor styling issues.
[1.3.0] - 2021-11-11¶
Added¶
Command to export and compress all the data.
Filter translated from to the Resource search.
Toggle to mark Resources as private. Private resources are only visible to authenticated users.
README file to the data export.
Event page type.
Changed¶
Rename the Resource export command to export_resources to make it consistent with the other export commands.
Store the exported data in the media directory.
Restore radicalism in subject filter.
Replace replaceAll with replace(//g) to increase browser support.
Fixed¶
Display of private search results for non-authenticated users.
Removed¶
Radical markers, they are not required by the research team anymore.
[1.2.4] - 2021-09-21¶
Fixed¶
Agent page links.
[1.2.3] - 2021-09-08¶
Changed¶
Do not export resource relationship ids, they are redundant.
Fixed¶
Add dates from the resources to the paratexts.
Range search.
[1.2.2] - 2021-07-14¶
Fixed¶
Update map view when filters change.
[1.2.1] - 2021-07-14¶
Changed¶
Bump Django and Wagtail.
Fixed¶
Map display.
Source text label on map popups.
[1.2.0] - 2021-07-14¶
Added¶
Comparative timeline to the home page.
Zoom feature to the timeline grid.
Sorting options to the base settings.
Wikidata as an option for Agent page.
Resources network visualisation.
Agents network visualisation.
Changed¶
Remove country names from the events classification filter.
Display related events in the events timeline.
Focus middle of the timeline.
Sort agents by last name.
Sort anonymous agents last.
Removed¶
Comparative value from the events classification filter.
Scatter events timeline.
Event fields from the Resource search because the data is not complete.
Fixed¶
Display of noble status.
Places filters.
Missing migration.
Centering the timeline on all browsers.
[1.1.3] - 2021-07-06¶
Changed¶
Key after changes by the research team.
Upgrade nginx image.
[1.1.2] - 2021-06-10¶
Fixed¶
Display of duplicate contributions.
[1.1.1] - 2021-05-26¶
Added¶
Published as filter.
Resource labels for has translation and has other edition.
Links to URLs in the bibliography.
Changed¶
Pseudonyms and ‘published as anonymous’ display in brackets after the author’s name.
Index contributions by agent’s name not pseudonyms.
Other editions that are not source-texts are marked as translations.
Fixed¶
Display of duplicate contributions.
Issue deleting contributions.
[1.1.0] - 2021-05-17¶
Added¶
Events visualisation.
Map clustering.
GitHub action for CI.
Map clustering.
Docs: data workflow diagram.
Blog tags navigation.
Radical markers to Resource detail view.
Radical markers to the search index and search results.
Add search options to the settings.
Place API view.
Load analytics setting.
GitHub action for CI.
Agents search.
Command to export Resources to a CSV file.
Command to export Organisations and Persons to CSV files.
Command to export Events to a CSV file.
Docs: data model fields concordance.
Command to export Places to a CSV file.
Convert notes and summary URLs into clickable links.
Badge for related resources that have a paratext.
Paratext functions vocabulary.
Radical date filter.
Changed¶
Add profiles to the pa11y, mongo and kibana services in local.yml.
Docs: update data models and architecture diagrams.
Add paratext term Revolutionary calendar use to Resources that have a radical date.
Modal map view uses an API call to get Place information rather than loading a Django view.
Upgrade Wagtail to version 2.12.
Exclude libraries from the list of Organisations.
Exclude radicalism from the subject filter.
Order/display Resources contributor filter by family name, given name.
Order/display of Contributions.
Labels and descriptions of the classification vocabularies.
Split paratext terms vocabulary in two, paratext forms and paratext functions.
Fixed¶
Validation errors in CHANGELOG.
Search results map loading delay.
Date interval parsing issues.
Search results map not updating.
Error when trying to add a map link to a publication place that is only a fictional place.
The ordering of reverse relationships.
Removed¶
Place detail view.
Radical marker from dates.
Radical markers filter from the search.
Revolutionary calendar used filter from paratext filters.
[1.0.0] - 2021-01-27¶
Added¶
Data migration to merge duplicate libraries (Organisation).
Field radical to Agent.
Data migration to merge duplicate paratext terms.
State management to search.
Help text to both filter types (main and secondary).
Help text to remove applied filters.
Map view to the search results.
Tag to source text search results.
Paratext search filter.
Modal view for Place references.
Changed¶
is_original to include resources that are marked both as original and source text.
Exclude the original/source text terms from being indexed with the translation terms.
Source text and translation moved to top level filters.
Split filters view into main and secondary.
Translation facet label to Translation Terms.
Move year facet to main filters section.
Order related resources by date, relationship and title.
Change Original to Source text.
Fixed¶
Reset pagination when filtering by year or updating filters.
[0.10.3] - 2021-01-12¶
Added¶
Histogram to range facets.
Click event to the histogram.
[0.10.2] - 2021-01-05¶
Added¶
Slider to range facets.
Analytics script.
[0.10.1] - 2020-12-08¶
Changed¶
Reduce the search fuzziness distance to 1.
Fixed¶
Do not display notes in bibliography page.
[0.10.0] - 2020-12-07¶
Added¶
Elasticsearch service.
Kibana service.
Resource search index.
Agents search index.
Events search index.
Resource text search with filters, ordering and pagination of search results.
Resource search tests.
Resource search suggestions.
Fuzzy text search.
Search results highlighting.
Search styling.
[0.9.1] - 2020-12-01¶
Fixed¶
Images formatting and alignment.
Icons display.
[0.9.0] - 2020-11-30¶
Added¶
Data migration to copy subjects from the translation resource to the original resource.
Pa11y dashboard service.
Data migration to merge duplicate libraries (Organisation).
Data migration to merge duplicate languages.
Function to get the date of a Resource.
Changed¶
Truncate breadcrumb to 5 words.
Truncate metatitle to 10 words.
Remove unused and duplicate packages.
Swap the labels for main and other places in the Agent detail view.
Fixed¶
Do not display tag icon if there are not tags associated with a blog post.
[0.8.0] - 2020-11-06¶
Added¶
Migration to convert author roles to translator when the Resource is a translation.
Field to record fictional places of publication.
Helper functions to Date, to get the earliest and latest dates for an object.
Migration to convert Essay term from FAST topics to FAST forms vocabulary.
Wagtail page type for the home page.
Wagtail page type for biographies.
Template tag to render breadcrumbs.
Changed¶
Upgrade Controlled Vocabulary application.
Upgrade Wagtail to version 2.9.
[0.7.1] - 2020-07-02¶
Added¶
CERL vocabulary for Agent models.
Docker Compose restart policies to the Docker services.
Django email configuration.
Fields main_places and noble to Person.
New application, cms, for Wagtail customisations.
Changed¶
Add date to Resource string for better disambiguation.
Simplify the Fabric commands.
Index page template to display extra information for blog posts.
Removed¶
Helper script, it has been replaced with the Fabric file.
Anymail integration.
django-allauth integration.
Fixed¶
TyperError in Resource __str__, was preventing the editing of records.
Error templates.
Admin favicon.
Issues with Controlled Vocabulary application.
[0.7.0] - 2020-06-17¶
Added¶
Resource views.
Agent views.
Event views.
Configuration for dev, stg, and liv instances.
Fabric script for remote task automation.
Zotero integration to harvest bibliographic data from Zotero.
[0.6.3] - 2020-06-08¶
Changed¶
Date display format to include radical date when available.
Prefix paratext Resources with [paratext].
Replace Classification source with editorial classification field.
Fixed¶
Update Django Controlled Vocabulary app.
Add missing vocabulary entry for Printing and Publishing Terms.
Autocomplete for Event and Place models.
Issue deleting Resource contributions.
[0.6.2] - 2020-06-02¶
Changed¶
Disable automatic conversion of dates.
[0.6.1] - 2020-06-02¶
Added¶
KDL Wagtail People page types.
Sources and notes fields to Agent.
Changed¶
Do not display French Republican dates by default.
Domain name, radicaltranslations.org.
Fixed¶
Agent search.
[0.6.0] - 2020-06-01¶
Added¶
Log entries to the admin interface.
wagtailmenus app.
Conversion from Gregorian to French Republican dates.
Command to import Resource URLs from GSX.
Basic styling and typography.
Changed¶
Wagtail now serves the root URL.
Agents admin, add extra search fields and filters.
Reorganise KDL Wagtail templates.
Fixed¶
Add missing Wagtail apps.
Resource, electronic_locator import.
[0.5.1] - 2020-05-27¶
Changed¶
When importing Resource check if a resource with the same title and date already exists.
Import Resource relationships after all the resources are imported to avoid conflicts.
[0.5.0] - 2020-05-27¶
Added¶
New tests for Resource.
New tests for Title.
Original as a value for Classification.edition vocabulary.
nginx to serve media files.
Changed¶
Update vocabularies with values provided by the research team.
[0.10.0] - 2020-12-07¶
Added¶
Elasticsearch service.
Kibana service.
Resource search index.
Agents search index.
Events search index.
Resource text search with filters, ordering and pagination of search results.
Resource search tests.
Resource search suggestions.
Fuzzy text search.
Search results highlighting.
Search styling.
[0.9.1] - 2020-12-01¶
Fixed¶
Images formatting and alignment.
Icons display.
[0.9.0] - 2020-11-30¶
Added¶
Data migration to copy subjects from the translation resource to the original resource.
Pa11y dashboard service.
Data migration to merge duplicate libraries (Organisation).
Data migration to merge duplicate languages.
Function to get the date of a Resource.
Changed¶
Truncate breadcrumb to 5 words.
Truncate metatitle to 10 words.
Remove unused and duplicate packages.
Swap the labels for main and other places in the Agent detail view.
Fixed¶
Do not display tag icon if there are not tags associated with a blog post.
[0.8.0] - 2020-11-06¶
Added¶
Migration to convert author roles to translator when the Resource is a translation.
Field to record fictional places of publication.
Helper functions to Date, to get the earliest and latest dates for an object.
Migration to convert Essay term from FAST topics to FAST forms vocabulary.
Wagtail page type for the home page.
Wagtail page type for biographies.
Template tag to render breadcrumbs.
Changed¶
Upgrade Controlled Vocabulary application.
Upgrade Wagtail to version 2.9.
[0.7.1] - 2020-07-02¶
Added¶
CERL vocabulary for Agent models.
Docker Compose restart policies to the Docker services.
Django email configuration.
Fields main_places and noble to Person.
New application, cms, for Wagtail customisations.
Changed¶
Add date to Resource string for better disambiguation.
Simplify the Fabric commands.
Index page template to display extra information for blog posts.
Removed¶
Helper script, it has been replaced with the Fabric file.
Anymail integration.
django-allauth integration.
Fixed¶
TyperError in Resource __str__, was preventing the editing of records.
Error templates.
Admin favicon.
Issues with Controlled Vocabulary application.
[0.7.0] - 2020-06-17¶
Added¶
Resource views.
Agent views.
Event views.
Configuration for dev, stg, and liv instances.
Fabric script for remote task automation.
Zotero integration to harvest bibliographic data from Zotero.
[0.6.3] - 2020-06-08¶
Changed¶
Date display format to include radical date when available.
Prefix paratext Resources with [paratext].
Replace Classification source with editorial classification field.
Fixed¶
Update Django Controlled Vocabulary app.
Add missing vocabulary entry for Printing and Publishing Terms.
Autocomplete for Event and Place models.
Issue deleting Resource contributions.
[0.6.2] - 2020-06-02¶
Changed¶
Disable automatic conversion of dates.
[0.6.1] - 2020-06-02¶
Added¶
KDL Wagtail People page types.
Sources and notes fields to Agent.
Changed¶
Do not display French Republican dates by default.
Domain name, radicaltranslations.org.
Fixed¶
Agent search.
[0.6.0] - 2020-06-01¶
Added¶
Log entries to the admin interface.
wagtailmenus app.
Conversion from Gregorian to French Republican dates.
Command to import Resource URLs from GSX.
Basic styling and typography.
Changed¶
Wagtail now serves the root URL.
Agents admin, add extra search fields and filters.
Reorganise KDL Wagtail templates.
Fixed¶
Add missing Wagtail apps.
Resource, electronic_locator import.
[0.5.1] - 2020-05-27¶
Changed¶
When importing Resource check if a resource with the same title and date already exists.
Import Resource relationships after all the resources are imported to avoid conflicts.
[0.5.0] - 2020-05-27¶
Added¶
New tests for Resource.
New tests for Title.
Original as a value for Classification.edition vocabulary.
nginx to serve media files.
Changed¶
Update vocabularies with values provided by the research team.
[0.4.1] - 2020-05-19¶
Fixed¶
Classification tests.
[0.4.0] - 2020-05-19¶
Changed¶
For simplicity the Work/Instance/Item objects have been flattened into Resource.
[0.3.1] - 2020-05-12¶
Fixed¶
Constraint on unique titles, it potentially caused duplicate entries under race conditions.
[0.3.0] - 2020-05-11¶
Added¶
Custom vocabulary for
Classification.edition
.Team information to the docs.
humans.txt (http://humanstxt.org/).
Place of birth and place of death to Person model.
Paratext mapping.
Date field to record dates that are in alternative formats.
New resource relationship types.
Editorial classification field to further specify relationships between objects.
Field for contributions under pseudonyms.
Chicago: Rare Books and Manuscripts Section controlled vocabulary.
Changed¶
The import Resource command to import Item records from GSX.
Reduce the number of models in the admin interface.
Allow part of relationships for original works.
Add counter as subtitle to imported Untitled and Translation titles from GSX.
Fixed¶
Production Django Dockerfile: add missing dependencies.
Resource
import, it was failing to import Original works.Resource
import, it was creatingWork
objects for derivative instances.Resource
import, import relationships to multipleWork
objects.
Security¶
Upgrade jQuery, https://blog.jquery.com/2020/04/10/jquery-3-5-0-released/
Upgrade Wagtail, https://docs.wagtail.io/en/stable/releases/2.8.1.html
[0.2.2] - 2020-02-12¶
Changed¶
Change the format of this file to adhere to Keep a Changelog.
Security¶
Bump Django from 2.2.9 to 2.2.10 (https://github.com/kingsdigitallab/radical_translations/pull/2)
[0.2.1] - 2020-02-11¶
Fixed¶
Import of resources with editions.
[0.2.0] - 2020-02-11¶
Added¶
Add command to import
Event
records from Google Spreadsheet JSON (GSX).Add command to import
Organisation
records from GSX.Add command to import
Person
records from GSX.Add command to import
Resource
records from GSX.
[0.1.0] - 2020-02-05¶
Added¶
Initial data models