Nov 25 - 27
9:00 a.m. - 3:00 p.m.
Instructors: Lucia Michielin, Matthew Hamilton, Mario Antonioletti, Geoff Lee
Helpers: Bailey Harrington, Robert Nagy, Jen Daub, Charlotte Desvages, Alejandro Delgado, David McKain
Data Carpentry develops and teaches workshops on the fundamental data skills needed to conduct research. Its target audience is researchers who have little to no prior computational experience, and its lessons are domain specific, building on learners' existing knowledge to enable them to quickly apply skills learned to their own research. Participants will be encouraged to help one another and to apply what they have learned to their own research problems.
For more information on what we teach and why, please see our paper "Good Enough Practices for Scientific Computing".
Who: The course is aimed at graduate students and other researchers. You don't need to have any previous knowledge of the tools that will be presented at the workshop.
Where: This training will take place online. The instructors will provide you with the information you will need to connect to this meeting.
When: Nov 25 - 27. Add to your Google Calendar.
Requirements: Participants must bring a laptop with a Mac, Linux, or Windows operating system (not a tablet, Chromebook, etc.) that they have administrative privileges on. They should have a few specific software packages installed (listed below).
Accessibility: We are dedicated to providing a positive and accessible learning environment for all. Please notify the instructors in advance of the workshop if you require any accommodations or if there is anything we can do to make this workshop more accessible to you.
Contact: Please email email@example.com for more information.
Roles: To learn more about the roles at the workshop (who will be doing what), refer to our Workshop FAQ.
Everyone who participates in Carpentries activities is required to conform to the Code of Conduct. This document also outlines how to report an incident if needed.
We will use this collaborative document for chatting, taking notes, and sharing URLs and bits of code.
Please be sure to complete these surveys before and after the workshop.
|Before starting||Pre-workshop survey|
|09:00||Data Organization in Spreadsheets|
|10:30||Morning break (15 min)|
|10:45||Data Organization in Spreadsheets|
|12:30||OpenRefine for Data Cleaning|
|09:00||Python for Humanities: Introduction|
|10:30||Morning break (15 min)|
|10:45||Python for Humanities: Continued|
|12:30||Python for Humanities: Continued|
|09:00||Python for Humanities: Continued|
|10:30||Morning break (15 min)|
|10:45||Python for Humanities: Conclusion|
|12:30||Data Management with SQL|
|After Finishing||Post-workshop survey|
To participate in a Data Carpentry workshop, you will need access to the software described below. In addition, you will need an up-to-date web browser.
We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.
If you haven't used Zoom before, go to the official website to download and install the Zoom client for your computer.
Like other Carpentries workshops, you will be learning by "coding along" with the Instructors. To do this, you will need to have both the window for the tool you will be learning about (a terminal, RStudio, your web browser, etc..) and the window for the Zoom video conference client open. In order to see both at once, we recommend using one of the following set up options:
A list of shortcuts for common Zoom functions you will use during the workshop.
|Start/stop screen share||⌘+⇧+S||Alt+Shift+S|
|Display/hide participants panel||⌘+U||Alt+U|
|Show/hide in-meeting chat panel||⌘+⇧+H||Alt+H|
|Raise hand/lower hand||⌥+Y||Alt+Y|
|Enter or exit full screen||⌘+⇧+F||Alt+F|
This workshop is designed to be run on your laptop. First, you will need to download the data we use in the workshop. Then, you need to install some software. After following the instructions on this page, you should have everything you need to participate fully in the workshop!
This will automatically download all of the files to your default download directory as a single compressed
.zip) file. To expand this file, double click the folder icon in your file navigator application (for Macs, this is the Finder application).
The data for this lesson is a part of the Data Carpentry Humanities workshop. It is a teaching version of the EEBO/TCP metadata catalogue. The data in this lesson is a subset of the teaching version that has been intentionally ‘messed up’ for this lesson.
|Spreadsheet program||Link||Link||Linux, MacOS, Windows||Spreadsheet program for organizing tabular data.|
|OpenRefine||Link||Link||Linux, MacOS, Windows|
|Python||See install instructions below.||Linux, MacOS, Windows|
To interact with spreadsheets, we can use LibreOffice, Microsoft Excel, Gnumeric, OpenOffice.org, or other programs. Commands may differ a bit between programs, but the general ideas for thinking about spreadsheets are the same. For this workshop, we recommend using either Microsoft Excel (paid software) or LibreOffice (free and open source). Other spreadsheet programs may not have all of the features we will be exploring in this workshop.
To install LibreOffice, go to their download page. The website should automatically select the correct option for your operating system. Click the “Download” button. You will go to a page that asks about a donation, but you don’t need to make one. Your download should begin automatically. Once the installer is downloaded, double click on it (you may need to open your Downloads folder) and LibreOffice should install.
OpenRefine is a Java program that runs on your local machine (not on the cloud). Although it displays in your browser, no web connection is needed and your data remains local. You need to have a ‘Java Runtime Environment’ (JRE) installed on your computer to run OpenRefine. If you don’t already have one installed then you can download and install from http://java.com by going to the site and clicking “Free Java Download”.
To install OpenRefine, go to their download page. From the download page, select either “Windows
kit”, “Mac kit”, or “Linux kit” - depending on your operating system - and follow the instructions next to your download link. This
lesson has been tested with all versions of OpenRefine up to the latest tested version, 3.2. If you are using an older version, it is
recommended you upgrade to the latest tested version. After installing, you can delete the installer
You may get an error message: “OpenRefine.app can’t be opened because it is from an unidentified developer.” If you get this message, open your system preferences and click “Security & Privacy”. You will see a message “OpenRefine.app was blocked from opening because it is from an unidentified developer.” Click “Open Anyway” and “Yes”. OpenRefine should open in your default web browser.
OpenRefine does not support Internet Explorer or Edge. Please use Firefox, Chrome or Safari instead.
This is a list of common operations you will use in your Jupyter Notebooks
|Run Cells and Select Below||⇧+⏎||SHIFT+ENTER|
|Run Cells and Insert Below||⌥+⏎||ALT+ENTER|
|Toggle Line Numbers||⇧+L||SHIFT+L|
|New Cell Above||a||a|
|New Cell Below||b||b|
|Delete Cell||d d||d d|
SQL is a specialized programming language used with databases. We use a simple database manager called SQLite in our lessons. We will use the DB Browser for SQLite program, which is available for all major platforms.
To install the DB Browser, go to their download page and choose the correct installer for
your operating system. Once the installer is downloaded, double click on it (you may need to open your Downloads folder), follow
any other instructions that appear, and
DB Browser should install. After installing, you can delete the installer
Congratulations! You are now ready for the workshop!