nbgrader¶

nbgrader is a tool that facilitates creating and grading assignments in the Jupyter notebook. It allows instructors to easily create notebook-based assignments that include both coding exercises and written free-responses. nbgrader then also provides a streamlined interface for quickly grading completed assignments.

For an overview and demonstration of nbgrader’s core functionality, check out the talk on nbgrader given at SciPy 2017:

For additional resources on using Jupyter in educational contexts, see the jupyter4edu ideas repository.

Highlights¶

Broadly speaking, nbgrader implements:

A Jupyter notebook format for assignments: completely normal Jupyter notebooks with metadata to make them useful for teaching.
Student interface as Jupyter extensions.
Instructor interface as Jupyter extensions.
A method of exchanging files between instructors and students.

Notebook format¶

Nbgrader actually doesn’t have its own notebook format: it is completely standard Jupyter, so you don’t have to learn anything new or need to put code into a teaching-only format. Let’s look at an example of an instructor’s notebook:

# Cell 1, cell metadata=autograded-answer
def add(a, b):
    ### BEGIN SOLUTION
    c = a + b
    ### END SOLUTION
    return c

# Cell 2, cell metadata=autograder-tests
assert add(1, 2) == 3
### BEGIN HIDDEN TESTS
assert add(-1, 2) == 1
### END HIDDEN TESTS

We see that this is a perfectly normal notebook, with cell metadata tagging cells and some ### markup indicating lines to be changed/removed in the student version. Testing the assignment assignment is as simple as running the entire notebook and looking for errors.

The student version is generated from the above to get:

# Cell 1
def add(a, b):
    # YOUR CODE HERE
    raise NotImplementedError()
    return c

# Cell 2
assert add(1, 2) == 3

We see that this is also a valid notebook, with the assignment parts replaced with a placeholder. The student can attempt to do the assignment, and checking their work is as simple as “Restart kernel and run all cells” and looking for errors (which the validate button implements automatically).

When the instructor receives the assignment back from the students, Cell 2 is restored to its initial values (restoring the hidden test while leaving the student solution), and the autograding implementation is as simple as “Run all cells” and looking for error output in the autograded-tests cells).

Student interface¶

Student assignment list extension for Jupyter notebooks¶

Using the assignment list extension, students may conveniently view, fetch, submit, and validate their assignments. This is also where they recieve and review any feedback on those submissions:

Instructor interface¶

Instructor “formgrader” extension for Jupyter notebooks¶

The formgrader extension for the Jupyter notebook allows instructors to use the core functionality of nbgrader—generating the student version of an assignment, releasing assignments to students, collecting assignments, autograding submissions, and manually grading submissions.

The command line tools of nbgrader¶

The command line tools offer an efficient way for the instructor to generate, assign, release, collect, and grade notebooks. Here are some of the commands:

nbgrader generate_assignment: create a student version of a notebook
nbgrader release_assignment: release a notebook to students
nbgrader collect: collect students’ submissions
nbgrader autograde: autograde students’ submissions
nbgrader generate_feedback: create feedback files from graded submissions
nbgrader release_feedback: release the feeback files to students

The command line also offers students a way of working with notebooks:

nbgrader fetch: gets a released notebook
nbgrader submit: deposit a notebook for grading/review
nbgrader fetch_feedback: get any feeback for a submission

The philosophy and the approach¶

The nbgrader project evolved from my experiences as an instructor and a student. This excerpt from David Marr’s book, Vision, is one of the core readings of the class that inspired the creation of nbgrader. Dr. Marr was an originator of the field of computational neuroscience and has been extremely influential in the field of computational cognitive science as well. On behalf of the many individuals who contribute to nbgrader, I hope you find this project enhances your teaching and learning experiences.

– Jess Hamrick, UC Berkeley

How to structure course files¶

For instructor ease of use and developer maintenance, nbgrader makes a few assumptions about how your assignment files are organized. By default, nbgrader assumes that your assignments will be organized with the following directory structure:

{course_directory}/{nbgrader_step}/{student_id}/{assignment_id}/{notebook_id}.ipynb

where:

course_directory variable is the root directory where you run the nbgrader commands. This means that you can place your class files directory wherever you want. However, this location can also be customized (see the configuration options) so that you can run the nbgrader commands from anywhere on your system, but still have them operate on the same directory.
nbgrader_step - Each subcommand of nbgrader (e.g. assign, autograde, etc.) has different input and output folders associated with it. These correspond to the nbgrader_step variable – for example, the default input step directory for nbgrader autograde is “submitted”, while the default output step directory is “autograded”.
student_id corresponds to the unique ID of a student.
assignment_id corresponds to the unique name of an assignment.
notebook_id corresponds to the name of a notebook within an assignment (excluding the .ipynb extension).

Example¶

Taking the autograde step as an example, when we run the command nbgrader autograde "Problem Set 1", nbgrader will look for all notebooks that match the following path:

submitted/*/Problem Set 1/*.ipynb

For each notebook that it finds, it will extract the student_id, assignment_id, and notebook_id according to the directory structure described above. It will then autograde the notebook, and save the autograded version to:

autograded/{student_id}/Problem Set 1/{notebook_id}.ipynb

where student_id and notebook_id were parsed from the input file path.

Here is how a sample directory structure for the course named course101 might look, where the users bitdiddle and hacker have submitted solutions to the assignment ps1:

course101/
├── gradebook.db
├── nbgrader_config.py
├── source
│   ├── header.ipynb
│   └── ps1
│       ├── jupyter.png
│       ├── problem1.ipynb
│       └── problem2.ipynb
├── release
│   └── ps1
│       ├── jupyter.png
│       ├── problem1.ipynb
│       └── problem2.ipynb
├── submitted
│   ├── bitdiddle
│   │   └── ps1
│   │       ├── jupyter.png
│   │       ├── problem1.ipynb
│   │       ├── problem2.ipynb
│   │       └── timestamp.txt
│   └── hacker
│       └── ps1
│           ├── jupyter.png
│           ├── problem1.ipynb
│           ├── problem2.ipynb
│           └── timestamp.txt
├── autograded/
└── feedback/

On the student side, the directory structure follows a similar structure:

course101/
├── ps1
│   ├── jupyter.png
│   ├── problem1.ipynb
│   ├── problem2.ipynb
│   └── feedback
│       ├── 2019-05-31-11-48-34
│       │   ├── problem1.html
│       │   └── problem2.html
│       └── 2019-05-30-19-23-45
│           ├── problem1.html
│           └── problem2.html
└── ps2

Database of assignments¶

Additionally, nbgrader needs access to a database to store information about the assignments. This database is, by default, a sqlite database that lives at {course_directory}/gradebook.db, but you can also configure this to be any location of your choosing. You do not need to manually create this database yourself, as nbgrader will create it for you, but you probably want to prepopulate it with some information about assignment due dates and students (see Managing the database).

Additionally, nbgrader uses SQLAlchemy, so you should be able to also use MySQL or PostgreSQL backends as well (though in these cases, you will need to create the database ahead of time, as this is just how MySQL and PostgreSQL work).

Configuration files¶

To customize nbgrader’s behavior, you can set configuration options in the nbgrader_config.py file. In particular, if you are using nbgrader with JupyterHub, you must set the course id in the nbgrader configuration file. A basic nbgrader config file would live at {course_directory}/nbgrader_config.py and might look like:

c = get_config()
c.CourseDirectory.course_id = "course101"

There are many additional options you can configure. See Configuration options for a full list.

Creating and grading assignments ¶

This guide walks an instructor through the workflow for generating an assignment and preparing it for release to students.

Table of Contents

Creating and grading assignments

New in version 0.5.0: Much of the core functionality of nbgrader can now be accessed through the “formgrader” extension.

Accessing the formgrader extension ¶

See also

Installation: Instructions on how to install the formgrader extension.

The formgrader extension provides the core access to nbgrader’s instructor tools. After the extension has been installed, you can access it through the tab in the notebook list:

Creating a new assignment ¶

See also

Managing the database: Instructions on how to manage assignments in the database from the command line
nbgrader db assignment add: Command line options for nbgrader db assignment add

From the formgrader¶

To create a new assignment, open the formgrader extension and click the “Add new assignment…” button at the bottom of the page. This will ask you to provide some information such as the name of the assignment and its due date. Then, you can add files to the assignment and edit them by clicking the name of the assignment:

From the command line¶

If you are not using the formgrader extension, you can add a new assignment simply by creating a folder in your course directory with the name of the assignment. You can specify the assignment metadata (such as the duedate) using the nbgrader db assignment command (see Managing the database).

To simplify this example, two notebooks of the assignment have already been stored in the source/ps1 folder:

Developing assignments with the assignment toolbar ¶

Note: As you are developing your assignments, you should save them into the source/{assignment_id}/ folder of the nbgrader hierarchy, where assignment_id is the name of the assignment you are creating (e.g. “ps1”).

See also

The philosophy and the approach: More details on how the nbgrader hierarchy is structured.
Customizing how the student version of an assignment looks: Instructions for customizing how the student version of the assignment looks.

Before you can begin developing assignments, you will need to actually install the nbgrader toolbar. If you do not have it installed, please first follow the installation instructions.

Once the toolbar has been installed, you should see it in the drop down “View -> Cell Toolbar” menu:

Selecting the “Create Assignment” toolbar will create a separate toolbar for each cell which by default will be a dropdown menu with the “-” item selected. For markdown cells, there are two additional options to choose from, either “Manually graded answer” or “Read-only”:

For code cells, there are four options to choose from, including “Manually graded answer”, “Autograded answer”, “Autograder tests”, and “Read-only”:

The following sections go into detail about the different cell types, and show cells that are taken from a complete example of an assignment generated with the nbgrader toolbar extension:

“Manually graded answer” cells¶

If you select the “Manually graded answer” option (available for both markdown and code cells), the nbgrader extension will mark that cell as a cell that contains an answer that must be manually graded by a human grader. Here is an example of a manually graded answer cell:

The most common use case for this type of cell is for written free-response answers (for example, which interpret the results of code that may have been written and/or executed above).

When you specify a manually graded answer, you must additionally tell nbgrader how many points the answer is worth, and an id for the cell. Additionally, when creating the release version of the assignment (see Generate and release an assignment), the bodies of answer cells will be replaced with a code or text stub indicating to the students that they should put their answer or solution there. Please see Customizing how the student version of an assignment looks for details on how to customize this behavior.

Note: the blue border only shows up when the nbgrader extension toolbar is active; it will not be visible to students.

“Manually graded task” cells¶

New in version 0.6.0.

If you select the “Manually graded task” option (available for markdown cells), the nbgrader extension will mark that cell as a cell that contains the description of a task that students have to perform. They must be manually graded by a human grader. Here is an example of a manually graded answer cell:

The difference with a manually graded answer is that the manually graded tasks cells are not edited by the student. A manually or automatically graded cell ask students to perform a task in one cell. A manually graded task asks students to perform a task with cells.

The common use case for this type of cell is for tasks that require the student to create several cells such as “Process the data and create a plot to illustrate your results.” or to contain notebook-wide tasks such as “adhere to the PEP8 style convention.”

Note: the blue border only shows up when the nbgrader extension toolbar is active; it will not be visible to students.

“Manually graded task” cells with mark scheme¶

New in version 0.6.0.

A mark scheme can be created through the use of a special syntax such as === BEGIN MARK SCHEME === and === END MARK SCHEME ===. The section of text between the two markers will be removed from the student version, but will be visible at the grading stage and in the feedback.

“Autograded answer” cells¶

If you select the “Autograded answer” option (available only for code cells), the nbgrader extension will mark that cell as a cell that contains an answer which will be autograded. Here is an example of an autograded graded answer cell:

As shown in the image above, solutions can be specified inline, through the use of a special syntax such as ### BEGIN SOLUTION and ### END SOLUTION. When creating the release version (see Generate and release an assignment), the region between the special syntax lines will be replaced with a code stub. If this special syntax is not used, then the entire contents of the cell will be replaced with the code stub. Please see Customizing how the student version of an assignment looks for details on how to customize this behavior.

Unlike manually graded answers, autograded answers aren’t worth any points: instead, the points for autograded answers are specified for the particular tests that grade those answers. See the next section for further details.

Note: the blue border only shows up when the nbgrader extension toolbar is active; it will not be visible to students.

“Autograder tests” cells¶

If you select the “Autograder tests” option (available only for code cells), the nbgrader extension will mark that cell as a cell that contains tests to be run during autograding. Here is an example of two test cells:

Test cells should contain assert statements (or similar). When run through nbgrader autograde (see Autograde assignments), the cell will pass if no errors are raised, and fail otherwise. You must specify the number of points that each test cell is worth; then, if the tests pass during autograding, students will receive the specified number of points, and otherwise will receive zero points.

The lock icon on the left side of the cell toolbar indicates that the tests are “read-only”. See the next section for further details on what this means.

For tips on writing autograder tests, see Autograding resources.

Note: the blue border only shows up when the nbgrader extension toolbar is active; it will not be visible to students.

“Autograder tests” cells with hidden tests¶

New in version 0.5.0.

Tests in “Autograder tests” cells can be hidden through the use of a special syntax such as ### BEGIN HIDDEN TESTS and ### END HIDDEN TESTS, for example:

When creating the release version (see Generate and release an assignment), the region between the special syntax lines will be removed. If this special syntax is not used, then the contents of the cell will remain as is. Please see Customizing how the student version of an assignment looks for details on how to customize this behavior.

Note

Keep in mind that wrapping all tests (for an “Autograder tests” cell) in this special syntax will remove all these tests in the release version and the students will only see a blank cell. It is recommended to have at least one or more visible tests, or a comment in the cell for the students to see.

These hidden tests are placed back into the “Autograder tests” cells when running nbgrader autograde (see Autograde assignments).

“Read-only” cells¶

If you select the “Read-only” option (available for both code and markdown cells), the nbgrader extension will mark that cell as one that cannot be modified. This is indicated by a lock icon on the left side of the cell toolbar:

However, this doesn’t actually mean that it is truly read-only when opened in the notebook. Instead, what it means is that during the nbgrader generate_assignment step (see Generate and release an assignment), the source of these cells will be recorded into the database. Then, during the nbgrader autograde step (see Autograde assignments), nbgrader will check whether the source of the student’s version of the cell has changed. If it has, it will replace the cell’s source with the version in the database, thus effectively overwriting any changes the student made.

New in version 0.4.0: Read-only cells (and test cells) are now truly read-only! However, at the moment this functionality will only work on the master version of the notebook (5.0.0.dev).

This functionality is particularly important for test cells, which are always marked as read-only. Because the mechanism for autograding is that students receive full credit if the tests pass, an easy way to get around this would be to simply delete or comment out the tests. This read-only functionality will reverse any such changes made by the student.

Validating the instructor version ¶

See also

nbgrader validate: Command line options for nbgrader validate

From the validate extension¶

Ideally, the solutions in the instructor version should be correct and pass all the test cases to ensure that you are giving your students tests that they can actually pass. To verify this is the case, you can use the validate extension:

If your assignment passes all the tests, you’ll get a success pop-up:

If it doesn’t pass all the tests, you’ll get a message telling you which cells failed:

From the command line¶

You can also validate assignments on the command line using the nbgrader validate command:

%%bash

nbgrader validate source/ps1/*.ipynb

[ValidateApp | WARNING] No nbgrader_config.py file found (rerun with --debug to see where nbgrader is looking)
[ValidateApp | INFO] Validating '[NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/ps1/problem1.ipynb'
[ValidateApp | INFO] Validating '[NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/ps1/problem2.ipynb'

Success! Your notebook passes all the tests.
Success! Your notebook passes all the tests.

Generate and release an assignment ¶

See also

nbgrader generate assignment: Command line options for nbgrader generate_assignment
The philosophy and the approach: Details about how the directory hierarchy is structured
Configuration options: Details on nbgrader_config.py

From the formgrader¶

After an assignment has been created with the assignment toolbar, you will want to generate the version that students will receive. You can do this from the formgrader by clicking the “generate” button:

This should succeed with a pop-up window containing log output:

From the command line¶

As described in The philosophy and the approach, you need to organize your files in a particular way. For releasing assignments, you should have the master copy of your files saved (by default) in the following source directory structure:

{course_directory}/source/{assignment_id}/{notebook_id}.ipynb

Note: The student_id is not included here because the source and release versions of the assignment are the same for all students.

After running nbgrader generate_assignment, the release version of the notebooks will be:

{course_directory}/release/{assignment_id}/{notebook_id}.ipynb

As a reminder, the instructor is responsible for distributing this release version to their students using their institution’s existing student communication and document distribution infrastructure.

When running nbgrader generate_assignment, the assignment name (which is “ps1”) is passed. We also specify a header notebook (source/header.ipynb) to prepend at the beginning of each notebook in the assignment. By default, this command should be run from the root of the course directory:

%%bash

nbgrader generate_assignment "ps1" --IncludeHeaderFooter.header=source/header.ipynb --force

[GenerateAssignmentApp | WARNING] No nbgrader_config.py file found (rerun with --debug to see where nbgrader is looking)
[GenerateAssignmentApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/./ps1/jupyter.png
[GenerateAssignmentApp | INFO] Updating/creating assignment 'ps1': {}
[GenerateAssignmentApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/problem1.ipynb
[GenerateAssignmentApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/ps1/problem1.ipynb
[GenerateAssignmentApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/problem2.ipynb
[GenerateAssignmentApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/ps1/problem2.ipynb
[GenerateAssignmentApp | INFO] Setting destination file permissions to 644

Preview the student version ¶

After generating the student version of assignment, you should preview it to make sure that it looks correct. You can do this from the formgrader extension by clicking the “preview” button:

Under the hood, there will be a new folder called release with the same structure as source. The release folder contains the actual release version of the assignment files:

If you are working on the command line, you may want to formally verify the student version as well. Ideally, all the tests should fail in the student version if the student hasn’t implemented anything. To verify that this is in fact the case, we can use the nbgrader validate --invert command:

%%bash

nbgrader validate --invert release/ps1/*.ipynb

[ValidateApp | WARNING] No nbgrader_config.py file found (rerun with --debug to see where nbgrader is looking)
[ValidateApp | INFO] Validating '[NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/ps1/problem1.ipynb'
[ValidateApp | INFO] Validating '[NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/ps1/problem2.ipynb'

Success! The notebook does not pass any tests.
Success! The notebook does not pass any tests.

If the notebook fails all the test cases, you should see the message “Success! The notebook does not pass any tests.”

Releasing files to students and collecting submissions ¶

See also

Exchanging assignment files: Guide to releasing and collecting submissions.
nbgrader release assignment: Command line options for nbgrader release_assignment
nbgrader collect: Command line options for nbgrader collect
The philosophy and the approach: Details about how the directory hierarchy is structured
Configuration options: Details on nbgrader_config.py

Note: the Managing Assignment Files Guide goes into greater depth on how to release and collect assignments, and the various options that are available to do you for doing so.

At this point you will be able to take the files in the release folder and distribute them to students. If you are using nbgrader with JupyterHub, you can do this with either with the formgrader extension or with the nbgrader release_assignment command (see Exchanging assignment files). Otherwise, you will need to do this manually.

Similarly, to collect submissions, you can do this either with the formgrader extension or with the nbgrader collect command. Otherwise, you will need to manually place submitted files into the submitted directory. As described in The philosophy and the approach, you need to organize your files in a particular way. For submitted assignments, you should have the submitted versions of students’ assignments organized as follows:

submitted/{student_id}/{assignment_id}/{notebook_id}.ipynb

Please note: Students must use version 3 or greater of the IPython/Jupyter notebook for nbgrader to work properly. If they are not using version 3 or greater, it is possible for them to delete cells that contain important metadata for nbgrader. With version 3 or greater, there is a feature in the notebook that prevents cells from being deleted. See this issue for more details.

To ensure that students have a recent enough version of the notebook, you can include a cell such as the following in each notebook of the assignment:

import IPython
assert IPython.version_info[0] >= 3, "Your version of IPython is too old, please update it."

Autograde assignments ¶

See also

nbgrader autograde: Command line options for nbgrader autograde
The philosophy and the approach: Details about how the directory hierarchy is structured
Configuration options: Details on nbgrader_config.py

In the following example, we have an assignment with two notebooks. There are two submissions of the assignment:

Submission 1:

Submission 2:

From the formgrader¶

You can autograde individual submissions from the formgrader directly. To do so, click on the the number of submissions in the “Manage Assignments” view:

This will take you to a new page where you can see all the submissions. For a particular submission, click the “autograde” button to autograde it:

After autograding completes, you will see a pop-up window with log output:

And back on the submissions screen, you will see that the status of the submission has changed to “needs manual grading” and there is now a reported score as well:

From the command line¶

We can run the autograder for all students at once from the command line:

%%bash

nbgrader autograde "ps1" --force

[AutogradeApp | WARNING] No nbgrader_config.py file found (rerun with --debug to see where nbgrader is looking)
[AutogradeApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/bitdiddle/ps1/timestamp.txt -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/timestamp.txt
[AutogradeApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/bitdiddle/ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/jupyter.png
[AutogradeApp | INFO] Creating/updating student with ID 'bitdiddle': {}
[AutogradeApp | INFO] SubmittedAssignment<ps1 for bitdiddle> submitted at [timestamp]
[AutogradeApp | INFO] Overwriting files with master versions from the source directory
[AutogradeApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/jupyter.png
[AutogradeApp | INFO] Sanitizing [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/bitdiddle/ps1/problem1.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/bitdiddle/ps1/problem1.ipynb
[AutogradeApp | WARNING] Attribute 'checksum' for cell correct_squares has changed! (should be: 8f41dd0f9c8fd2da8e8708d73e506b3a, got: 845d4666cabb30b6c75fc534f7375bf5)
[AutogradeApp | WARNING] Attribute 'checksum' for cell squares_invalid_input has changed! (should be: 23c2b667d3b60eff3be46eb3290a6b4a, got: 123394e73f33a622ec057e2eae51a54a)
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem1.ipynb
[AutogradeApp | INFO] Autograding [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem1.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem1.ipynb
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem1.ipynb
[AutogradeApp | INFO] Sanitizing [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/bitdiddle/ps1/problem2.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/bitdiddle/ps1/problem2.ipynb
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem2.ipynb
[AutogradeApp | INFO] Autograding [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem2.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem2.ipynb
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem2.ipynb
[AutogradeApp | INFO] Setting destination file permissions to 444
[AutogradeApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/hacker/ps1/timestamp.txt -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/timestamp.txt
[AutogradeApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/hacker/ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/jupyter.png
[AutogradeApp | INFO] Creating/updating student with ID 'hacker': {}
[AutogradeApp | INFO] SubmittedAssignment<ps1 for hacker> submitted at [timestamp]
[AutogradeApp | INFO] Overwriting files with master versions from the source directory
[AutogradeApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/jupyter.png
[AutogradeApp | INFO] Sanitizing [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/hacker/ps1/problem1.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/hacker/ps1/problem1.ipynb
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem1.ipynb
[AutogradeApp | INFO] Autograding [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem1.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem1.ipynb
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem1.ipynb
[AutogradeApp | INFO] Sanitizing [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/hacker/ps1/problem2.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted/hacker/ps1/problem2.ipynb
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem2.ipynb
[AutogradeApp | INFO] Autograding [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem2.ipynb
[AutogradeApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem2.ipynb
[AutogradeApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem2.ipynb
[AutogradeApp | INFO] Setting destination file permissions to 444

When grading the submission for Bitdiddle, you’ll see some warnings that look like “Checksum for grade cell correct_squares has changed!”. What’s happening here is that nbgrader has recorded what the original contents of the grade cell correct_squares (when nbgrader generate_assignment was run), and is checking the submitted version against this original version. It has found that the submitted version changed (perhaps this student tried to cheat by commenting out the failing tests), and has therefore overwritten the submitted version of the tests with the original version of the tests.

You may also notice that there is a note saying “ps1 for Bitdiddle is 21503.948203 seconds late”. What is happening here is that nbgrader is detecting a file in Bitdiddle’s submission called timestamp.txt, reading in that timestamp, and saving it into the database. From there, it can compare the timestamp to the duedate of the problem set, and compute whether the submission is at all late.

Once the autograding is complete, there will be new directories for the autograded versions of the submissions:

autograded/{student_id}/{assignment_id}/{notebook_id}.ipynb

Autograded submission 1:

Autograded submission 2:

Manual grading ¶

See also

The philosophy and the approach: More details on how the nbgrader hierarchy is structured.
Configuration options: Details on nbgrader_config.py

After assignments have been autograded, they will saved into an autograded directory (see The philosophy and the approach for details):

After running nbgrader autograde, the autograded version of the notebooks will be:

autograded/{student_id}/{assignment_id}/{notebook_id}.ipynb

We can manually grade assignments through the formgrader as well, by clicking on the “Manual Grading” navigation button. This will provide you with an interface for hand grading assignments that it finds in the directory listed above. Note that this applies to all assignments as well – as long as the autograder has been run on the assignment, it will be available for manual grading via the formgrader.

Generate feedback on assignments ¶

See also

nbgrader generate feedback: Command line options for nbgrader generate_feedback
nbgrader release feedback: Command line options for nbgrader release_feedback
The philosophy and the approach: Details about how the directory hierarchy is structured
Configuration options: Details on nbgrader_config.py

As mentioned before, after assignments have been autograded and/or manually graded, they will located in the autograded directory (see The philosophy and the approach for details):

autograded/{student_id}/{assignment_id}/{notebook_id}.ipynb

Creating feedback for students is divided into two parts:

generate feedback
release feedback

Generating feedback will create HTML files in the local instructor directory. Releasing feedback will copy those HTML files to the nbgrader exchange.

We can generate feedback based on the graded notebooks by running the nbgrader generate_feedback command, which will produce HTML versions of these notebooks at the following location:

feedback/{student_id}/{assignment_id}/{notebook_id}.html

The nbgrader generate_feedback is available by clicking the Generate Feedback button on either the Manage Assignments view (to generate feedback for all graded submissions), or on the individual student’s Manage Submission page (to generate feedback for that specific individual).

We can release the generated feedback by running the nbgrader release_feedback command, which will send the generated HTML files to the nbgrader exchange.

The nbgrader release_feedback is available by clicking the Release Feedback button on either the Manage Assignments view (to release feedback for all generated feedback), or on the individual student’s Manage Submission page (to release feedback for that specific individual).

Workflow example: Instructor returning feedback to students¶

In some scenarios, you may not want to (or be able to) use the exchange to deliver student feedback. This sections describes a workflow for manually returning generated feedback.

In the following example, we have an assignment with two notebooks. There are two submissions of the assignment that have been graded:

Autograded submission 1:

Autograded submission 2:

Generating feedback is fairly straightforward (and as with the other nbgrader commands for instructors, this must be run from the root of the course directory):

%%bash

nbgrader generate_feedback "ps1"

[GenerateFeedbackApp | WARNING] No nbgrader_config.py file found (rerun with --debug to see where nbgrader is looking)
[GenerateFeedbackApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/timestamp.txt -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/bitdiddle/ps1/timestamp.txt
[GenerateFeedbackApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/bitdiddle/ps1/jupyter.png
[GenerateFeedbackApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem1.ipynb
[GenerateFeedbackApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/bitdiddle/ps1/problem1.html
[GenerateFeedbackApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/bitdiddle/ps1/problem2.ipynb
[GenerateFeedbackApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/bitdiddle/ps1/problem2.html
[GenerateFeedbackApp | INFO] Setting destination file permissions to 644
[GenerateFeedbackApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/timestamp.txt -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/hacker/ps1/timestamp.txt
[GenerateFeedbackApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/hacker/ps1/jupyter.png
[GenerateFeedbackApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem1.ipynb
[GenerateFeedbackApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/hacker/ps1/problem1.html
[GenerateFeedbackApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/autograded/hacker/ps1/problem2.ipynb
[GenerateFeedbackApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/feedback/hacker/ps1/problem2.html
[GenerateFeedbackApp | INFO] Setting destination file permissions to 644

Once the feedback has been generated, there will be new directories and HTML files corresponding to each notebook in each submission:

Feedback for submission 1:

Feedback for submission 2:

If the exchange is available, one would of course use nbgrader release_feedback. However if not available, you can now deliver these generated HTML feedback files via whatever mechanism you wish.

Getting grades from the database ¶

New in version 0.4.0.

See also

nbgrader export: Command line options for nbgrader export
Export plugin: Details on how to write your own custom exporter.

In addition to creating feedback for the students, you may need to upload grades to whatever learning management system your school uses (e.g. Canvas, Blackboard, etc.). nbgrader provides a way to export grades to CSV out of the box, with the nbgrader export command:

%%bash

nbgrader export

[ExportApp | WARNING] No nbgrader_config.py file found (rerun with --debug to see where nbgrader is looking)
[ExportApp | INFO] Using exporter: CsvExportPlugin
[ExportApp | INFO] Exporting grades to grades.csv

After running nbgrader export, you will see the grades in a CSV file called grades.csv:

%%bash

cat grades.csv

assignment,duedate,timestamp,student_id,last_name,first_name,email,raw_score,late_submission_penalty,score,max_score
ps1,,[timestamp],bitdiddle,,,,1.5,0.0,1.5,13.0
ps1,,[timestamp],hacker,,,,3.0,0.0,3.0,13.0

If you need to customize how the grades are exported, you can write your own exporter.

Managing the database¶

See also

nbgrader db student add: Command line options for nbgrader db student add
nbgrader db student import: Command line options for nbgrader db student import
nbgrader db student remove: Command line options for nbgrader db student remove
nbgrader db student list: Command line options for nbgrader db student list
nbgrader db assignment add: Command line options for nbgrader db assignment add
nbgrader db assignment import: Command line options for nbgrader db assignment import
nbgrader db assignment remove: Command line options for nbgrader db assignment remove
nbgrader db assignment list: Command line options for nbgrader db assignment list
The philosophy and the approach: More details on how the nbgrader hierarchy is structured.
Configuration options: Details on nbgrader_config.py

Most of the important information that nbgrader has access to—information about students, assignments, grades, etc.—is stored in the nbgrader database. Much of this is added to the database automatically by nbgrader, with the exception of two types of information: which students are in your class, and which assignments you have.

There are two methods for adding students and assignments to the database.

The first is by writing a Python script and using the API. The second way is to use the command line tool nbgrader db, which provides limited command line access to some of the API functionality.

%%bash

# remove the existing database, to start fresh
rm gradebook.db

Managing assignments¶

To add assignments, we can use the nbgrader db assignment add command, which takes the name of the assignment as well as optional arguments (such as its due date):

%%bash

nbgrader db assignment add ps1 --duedate="2015-02-02 17:00:00 UTC"

[DbAssignmentAddApp | INFO] Creating/updating assignment with ID 'ps1': {'duedate': '2015-02-02 17:00:00 UTC'}

After we have added the assignment, we can view what assignments exist in the database with nbgrader db assignment list:

%%bash

nbgrader db assignment list

There are 1 assignments in the database:
ps1 (due: 2015-02-02 17:00:00)

An alternate way to add assignments is a batch method of importing a CSV file. The file must have a column called name, and may optionally have columns for other assignment properties (such as the due date):

%%file assignments.csv
name,duedate
ps1,2015-02-02 17:00:00 UTC
ps2,2015-02-09 17:00:00 UTC

Writing assignments.csv

Then, to import this file, we use the nbgrader db assignment import command:

%%bash

nbgrader db assignment import assignments.csv

[DbAssignmentImportApp | INFO] Importing from: 'assignments.csv'
[DbAssignmentImportApp | INFO] Creating/updating Assignment with name 'ps1': {'duedate': '2015-02-02 17:00:00 UTC'}
[DbAssignmentImportApp | INFO] Creating/updating Assignment with name 'ps2': {'duedate': '2015-02-09 17:00:00 UTC'}

We can also remove assignments from the database with nbgrader db assignment remove. Be very careful using this command, as it is possible you could lose data!

%%bash

nbgrader db assignment remove ps1

[DbAssignmentRemoveApp | INFO] Removing assignment with ID 'ps1'

Managing students¶

Managing students in the database works almost exactly the same as managing assignments. To add students, we use the nbgrader db student add command:

%%bash

nbgrader db student add bitdiddle --last-name=Bitdiddle --first-name=Ben
nbgrader db student add hacker --last-name=Hacker --first-name=Alyssa

[DbStudentAddApp | INFO] Creating/updating student with ID 'bitdiddle': {'last_name': 'Bitdiddle', 'first_name': 'Ben', 'email': None, 'lms_user_id': None}
[DbStudentAddApp | INFO] Creating/updating student with ID 'hacker': {'last_name': 'Hacker', 'first_name': 'Alyssa', 'email': None, 'lms_user_id': None}

And to list the students in the database, we use the nbgrader db student list command:

%%bash

nbgrader db student list

There are 2 students in the database:
bitdiddle (Bitdiddle, Ben) -- None, None
hacker (Hacker, Alyssa) -- None, None

Like with the assignments, we can also batch add students to the database using the nbgrader db student import command. We first have to create a CSV file, which is required to have a column for id, and optionally may have columns for other student information (such as their name):

%%file students.csv
id,last_name,first_name,email
bitdiddle,Bitdiddle,Ben,
hacker,Hacker,Alyssa,

Writing students.csv

%%bash

nbgrader db student import students.csv

[DbStudentImportApp | INFO] Importing from: 'students.csv'
[DbStudentImportApp | INFO] Creating/updating Student with id 'bitdiddle': {'last_name': 'Bitdiddle', 'first_name': 'Ben', 'email': None}
[DbStudentImportApp | INFO] Creating/updating Student with id 'hacker': {'last_name': 'Hacker', 'first_name': 'Alyssa', 'email': None}

We can also remove students from the database with nbgrader db student remove. Be very careful using this command, as it is possible you could lose data!

%%bash

nbgrader db student remove bitdiddle

[DbStudentRemoveApp | INFO] Removing student with ID 'bitdiddle'

Exchanging assignment files ¶

Distributing assignments to students and collecting them can be a logistical nightmare. If you are running nbgrader on a server, some of this pain can be relieved by relying on nbgrader’s built-in functionality for releasing and collecting assignments on the instructor’s side, and fetching and submitting assignments on the student’s side.

This page describes the built-in implementation of an exchange directory coupled with instructor and student interfaces - both integrated in the Jupyter interface and via the command line. Since nbgrader 0.7.0, the exchange is modular, and a different implementation could be used (with the same user interface as below).

Table of Contents

Exchanging assignment files

Setting up the exchange ¶

After an assignment has been created using nbgrader generate_assignment, the instructor must actually release that assignment to students. If the class is being taught on a single filesystem, then the instructor may use nbgrader release_assignment to copy the assignment files to a shared location on the filesystem for students to then download.

First, we must specify a few configuration options. To do this, we’ll create a nbgrader_config.py file that will get automatically loaded when we run nbgrader:

%%file nbgrader_config.py

c = get_config()

c.CourseDirectory.course_id = "example_course"
c.Exchange.root = "/tmp/exchange"

Writing nbgrader_config.py

In the config file, we’ve specified the “exchange” directory to be /tmp/exchange. This directory must exist before running nbgrader, and it must be readable and writable by all users, so we’ll first create it and configure the appropriate permissions:

%%bash

# remove existing directory, so we can start fresh for demo purposes
rm -rf /tmp/exchange

# create the exchange directory, with write permissions for everyone
mkdir /tmp/exchange
chmod ugo+rw /tmp/exchange

Releasing assignments ¶

See also

Creating and grading assignments: Details on generating assignments
nbgrader release assignment: Command line options for nbgrader release_assignment
nbgrader list: Command line options for nbgrader list
The philosophy and the approach: More details on how the nbgrader hierarchy is structured.
Configuration options: Details on nbgrader_config.py

From the formgrader¶

Using the formgrader extension, you may release assignments by clicking on the “release” button:

Note that for the “release” button to become available, the course_id option must be set in nbgrader_config.py. Once completed, you will see a pop-up window with log output:

If you decide you want to “un-release” an assignment, you may do so by clicking again on the “release” button (which is now an “x”). However, note that students who have already downloaded the assignment will still have access to their downloaded copy. Unreleasing an assignment only prevents more students from downloading it.

From the command line¶

Now that we have the directory created, we can actually run nbgrader release_assignment (and as with the other nbgrader commands for instructors, this must be run from the root of the course directory):

%%bash

nbgrader release_assignment "ps1"

[ReleaseAssignmentApp | INFO] Source: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/./ps1
[ReleaseAssignmentApp | INFO] Destination: /tmp/exchange/example_course/outbound/ps1
[ReleaseAssignmentApp | INFO] Released as: example_course ps1

Finally, you can verify that the assignment has been appropriately released by running the nbgrader list command:

%%bash

nbgrader list

[ListApp | INFO] Released assignments:
[ListApp | INFO] example_course ps1

Note that there should only ever be one instructor who runs the nbgrader release_assignment and nbgrader collect commands (and there should probably only be one instructor – the same instructor – who runs nbgrader generate_assignment, nbgrader autograde and the formgrader as well). However this does not mean that only one instructor can do the grading, it just means that only one instructor manages the assignment files. Other instructors can still perform grading by accessing the notebook where the formgrader is running.

Fetching assignments ¶

See also

nbgrader fetch assignment: Command line options for nbgrader fetch_assignment
nbgrader list: Command line options for nbgrader list
Configuration options: Details on nbgrader_config.py

From the student’s perspective, they can list what assignments have been released, and then fetch a copy of the assignment to work on. First, we’ll create a temporary directory to represent the student’s home directory:

%%bash

# remove the fake student home directory if it exists, for demo purposes
rm -rf /tmp/student_home

# create the fake student home directory and switch to it
mkdir /tmp/student_home

If you are not using the default exchange directory (as is the case here), you will additionally need to provide your students with a configuration file that sets the appropriate directory for them:

%%file /tmp/student_home/nbgrader_config.py

c = get_config()
c.Exchange.root = '/tmp/exchange'
c.CourseDirectory.course_id = "example_course"

Writing /tmp/student_home/nbgrader_config.py

From the notebook dashboard¶

Warning

The “Assignment List” extension is not fully compatible with multiple courses on the same server. Please see Can I use the “Assignment List” extension with multiple classes? for details.

Alternatively, students can fetch assignments using the assignment list notebook server extension. You must have installed the extension by following the instructions here, after which you should see an “Assignments” tab in dashboard:

The image above shows that there has been one assignment released (“ps1”) for the class “example_course”. To get this assignment, students can click the “Fetch” button (analogous to running nbgrader fetch_assignment ps1 --course example_course. Note: this assumes nbgrader is always run from the root of the notebook server, which on JupyterHub is most likely the root of the user’s home directory.

After the assignment is fetched, it will appear in the list of “Downloaded assignments”:

Students can click on the name of the assignment to expand it and see all the notebooks in the assignment:

Clicking on a particular notebook will open it in a new tab in the browser.

From the command line¶

From the student’s perspective, they can see what assignments have been released using nbgrader list, and passing the name of the class:

%%bash
export HOME=/tmp/student_home && cd $HOME

nbgrader list

[ListApp | INFO] Released assignments:
[ListApp | INFO] example_course ps1

They can then fetch an assignment for that class using nbgrader fetch_assignment and passing the name of the class and the name of the assignment:

%%bash
export HOME=/tmp/student_home && cd $HOME

nbgrader fetch_assignment "ps1"

[FetchAssignmentApp | INFO] Source: /tmp/exchange/example_course/outbound/ps1
[FetchAssignmentApp | INFO] Destination: /tmp/student_home/ps1
[FetchAssignmentApp | INFO] Fetched as: example_course ps1

Note that running nbgrader fetch_assignment copies the assignment files from the exchange directory to the local directory, and therefore can be used from any directory:

%%bash

ls -l "/tmp/student_home/ps1"

total ##
-rw-r--r-- 1 nb_user nb_group [size] [date] [time] jupyter.png
-rw-r--r-- 1 nb_user nb_group [size] [date] [time] problem1.ipynb
-rw-r--r-- 1 nb_user nb_group [size] [date] [time] problem2.ipynb

Additionally, the nbgrader fetch_assignment (as well as nbgrader submit) command also does not rely on having access to the nbgrader database – the database is only used by instructors.

Submitting assignments ¶

See also

nbgrader submit: Command line options for nbgrader submit
nbgrader list: Command line options for nbgrader list
Configuration options: Details on nbgrader_config.py

From the notebook dashboard¶

Warning

The “Assignment List” extension is not fully compatible with multiple courses on the same server. Please see Can I use the “Assignment List” extension with multiple classes? for details.

Alternatively, students can submit assignments using the assignment list notebook server extension. You must have installed the extension by following the instructions here. Students must have also downloaded the assignments (see Fetching assignments).

After students have worked on the assignment for a while, but before submitting, they can validate that their notebooks pass the tests by clicking the “Validate” button (analogous to running nbgrader validate). If any tests fail, they will see a warning:

If there are no errors, they will see that the validation passes:

Note

If the notebook has been released with hidden tests removed from the source version (see “Autograder tests” cells with hidden tests) then this validation is only done against the tests the students can see in the release version.

Once students have validated all the notebooks, they can click the “Submit” button to submit the assignment (analogous to running nbgrader submit ps1 --course example_course). Afterwards, it will show up in the list of submitted assignments (and also still in the list of downloaded assignments):

Students may submit an assignment as many times as they’d like. All copies of a submission will show up in the submitted assignments list, and when the instructor collects the assignments, they will get the most recent version of the assignment:

Similarly, if the strict option (in the student’s nbgrader_config.py file) is set to True, the students will not be able to submit an assignment with missing notebooks (for a given assignment):

From the command line¶

First, as a reminder, here is what the student’s nbgrader_config.py file looks like:

%%bash

cat /tmp/student_home/nbgrader_config.py

c = get_config()
c.Exchange.root = '/tmp/exchange'
c.CourseDirectory.course_id = "example_course"

After working on an assignment, the student can submit their version for grading using nbgrader submit and passing the name of the assignment and the name of the class:

%%bash
export HOME=/tmp/student_home && cd $HOME

nbgrader submit "ps1"

[SubmitApp | INFO] Source: /tmp/student_home/ps1
[SubmitApp | INFO] Destination: /tmp/exchange/example_course/inbound/nb_user+ps1+[timestamp] UTC+[random string]
[SubmitApp | INFO] Submitted as: example_course ps1 [timestamp] UTC

Note that “the name of the assignment” really corresponds to “the name of a folder”. It just happens that, in our current directory, there is a folder called “ps1”:

%%bash
export HOME=/tmp/student_home && cd $HOME

ls -l "/tmp/student_home"

total ##
-rw-r--r-- 1 nb_user nb_group [size] [date] [time] nbgrader_config.py
drwxr-xr-x 1 nb_user nb_group [size] [date] [time] ps1

Students can see what assignments they have submitted using nbgrader list --inbound:

%%bash
export HOME=/tmp/student_home && cd $HOME

nbgrader list --inbound

[ListApp | INFO] Submitted assignments:
[ListApp | INFO] example_course nb_user ps1 [timestamp] UTC (no feedback available)

Importantly, students can run nbgrader submit as many times as they want, and all submitted copies of the assignment will be preserved:

%%bash
export HOME=/tmp/student_home && cd $HOME

nbgrader submit "ps1"

[SubmitApp | INFO] Source: /tmp/student_home/ps1
[SubmitApp | INFO] Destination: /tmp/exchange/example_course/inbound/nb_user+ps1+[timestamp] UTC+[random string]
[SubmitApp | INFO] Submitted as: example_course ps1 [timestamp] UTC

We can see all versions that have been submitted by again running nbgrader list --inbound:

%%bash
export HOME=/tmp/student_home && cd $HOME

nbgrader list --inbound

[ListApp | INFO] Submitted assignments:
[ListApp | INFO] example_course nb_user ps1 [timestamp] UTC (no feedback available)
[ListApp | INFO] example_course nb_user ps1 [timestamp] UTC (no feedback available)

Note that the nbgrader submit (as well as nbgrader fetch_assignment) command also does not rely on having access to the nbgrader database – the database is only used by instructors.

nbgrader requires that the submitted notebook names match the released notebook names for each assignment. For example if a student were to rename one of the given assignment notebooks:

%%bash
export HOME=/tmp/student_home && cd $HOME

# assume the student renamed the assignment file
mv ps1/problem1.ipynb ps1/myproblem1.ipynb

nbgrader submit "ps1"

[SubmitApp | INFO] Source: /tmp/student_home/ps1
[SubmitApp | INFO] Destination: /tmp/exchange/example_course/inbound/nb_user+ps1+[timestamp] UTC+[random string]
[SubmitApp | WARNING] Possible missing notebooks and/or extra notebooks submitted for assignment ps1:
    Expected:
            problem1.ipynb: MISSING
            problem2.ipynb: FOUND
    Submitted:
            myproblem1.ipynb: EXTRA
            problem2.ipynb: OK
[SubmitApp | INFO] Submitted as: example_course ps1 [timestamp] UTC

By default this assignment will still be submitted however only the “FOUND” notebooks (for the given assignment) can be autograded and will appear on the formgrade extension. “EXTRA” notebooks will not be autograded and will not appear on the formgrade extension.

To ensure that students cannot submit an assignment with missing notebooks (for a given assignment) the strict option, in the student’s nbgrader_config.py file, can be set to True:

%%file /tmp/student_home/nbgrader_config.py

c = get_config()
c.Exchange.root = '/tmp/exchange'
c.CourseDirectory.course_id = "example_course"
c.ExchangeSubmit.strict = True

Overwriting /tmp/student_home/nbgrader_config.py

%%bash
export HOME=/tmp/student_home && cd $HOME

nbgrader submit "ps1" || true

[SubmitApp | INFO] Source: /tmp/student_home/ps1
[SubmitApp | INFO] Destination: /tmp/exchange/example_course/inbound/nb_user+ps1+[timestamp] UTC+[random string]
[SubmitApp | CRITICAL] Assignment ps1 not submitted. There are missing notebooks for the submission:
    Expected:
            problem1.ipynb: MISSING
            problem2.ipynb: FOUND
    Submitted:
            myproblem1.ipynb: EXTRA
            problem2.ipynb: OK
[SubmitApp | ERROR] nbgrader submit failed

Collecting assignments ¶

See also

Creating and grading assignments: Details on grading assignments after they have been collected
nbgrader collect: Command line options for nbgrader collect
nbgrader list: Command line options for nbgrader list
The philosophy and the approach: More details on how the nbgrader hierarchy is structured.
Configuration options: Details on nbgrader_config.py

First, as a reminder, here is what the instructor’s nbgrader_config.py file looks like:

%%bash

cat nbgrader_config.py

c = get_config()

c.CourseDirectory.course_id = "example_course"
c.Exchange.root = "/tmp/exchange"

From the formgrader¶

From the formgrader extension, we can collect submissions by clicking on the “collect” button:

As with releasing, this will display a pop-up window when the operation is complete, telling you how many submissions were collected:

From here, you can click on the number of submissions to grade the collected submissions:

From the command line¶

After students have submitted their assignments, the instructor can view what has been submitted with nbgrader list --inbound:

%%bash

nbgrader list --inbound

[ListApp | INFO] Submitted assignments:
[ListApp | INFO] example_course nb_user ps1 [timestamp] UTC (no feedback available)
[ListApp | INFO] example_course nb_user ps1 [timestamp] UTC (no feedback available)
[ListApp | INFO] example_course nb_user ps1 [timestamp] UTC (no feedback available)

The instructor can then collect all submitted assignments with nbgrader collect and passing the name of the assignment (and as with the other nbgrader commands for instructors, this must be run from the root of the course directory):

%%bash

nbgrader collect "ps1"

[CollectApp | INFO] Processing 1 submissions of 'ps1' for course 'example_course'
[CollectApp | INFO] Collecting submission: nb_user ps1

This will copy the student submissions to the submitted folder in a way that is automatically compatible with nbgrader autograde:

%%bash

ls -l submitted

total ##
drwxr-xr-x 1 nb_user nb_group [size] [date] [time] bitdiddle
drwxr-xr-x 1 nb_user nb_group [size] [date] [time] hacker
drwxr-xr-x 1 nb_user nb_group [size] [date] [time] nb_user

Note that there should only ever be one instructor who runs the nbgrader release_assignment and nbgrader collect commands (and there should probably only be one instructor – the same instructor – who runs nbgrader generate_assignment, nbgrader autograde and the formgrader as well). However this does not mean that only one instructor can do the grading, it just means that only one instructor manages the assignment files. Other instructors can still perform grading by accessing the notebook server running the formgrader.

Exchanging assignment files manually ¶

After an assignment has been created using nbgrader generate_assignment, the instructor must actually release that assignment to students. This page describes how to do that using your institution’s existing learning management system, assuming that the students will fetch the assignments from - and submit their assignments to - the learning management system.

If this is not the case and you are using nbgrader in a shared server environment (e.g. JupyterHub), you can do this with an exchange implementation (see :doc:managing_assignment_files).

Distributing assignments to students and collecting them can be a logistical nightmare. The previous page discussed the built-in exchange directory, but that is not the only option (and in fact, was added later on). One can also distribute and collect files by other means, such as though your institution’s learning management system. If you are relying on your institution’s learning management system to get the submitted versions of notebooks back from students, nbgrader has some built-in functionality to make theat easier (putting the files in the right place into the course directory via an importer).

One can also do this fully manually, by sending files around. This may be useful during the testing phase.

Table of Contents

Exchanging assignment files manually

Releasing assignments ¶

In short, to release an assignment, send the files at release/{assignment_id}/* to your students. For example, you might post the files on your course page.

Submitting assignments ¶

When an assignment is submitted, it needs to be placed in submitted/{student_id}/{assignment_id}/*. The rest of this page describes the built-in ways to do this, if students upload them to a learning management system and you can download them all at once in an archive. This is called collecting the assignment.

Collecting assignments ¶

See also

nbgrader zip collect: Command line options for nbgrader zip_collect
ZipCollect plugins: Plugins for nbgrader zip_collect
The philosophy and the approach: More details on how the nbgrader hierarchy is structured.
Configuration options: Details on nbgrader_config.py

Once the students have submitted their assignments and you have downloaded these assignment files from your institution’s learning management system, you can get theses files back into nbgrader by using the nbgrader zip_collect sub-command.

Directory Structure:¶

nbgrader zip_collect makes a few assumptions about how the downloaded assignment files are organized. By default, nbgrader zip_collect assumes that your downloaded assignment files will be organized with the following directory structure:

{downloaded}/{assignment_id}/{collect_step}/

where:

The downloaded directory is the main directory where your downloaded assignment files are placed. This location can also be customized (see the configuration options) so that you can run the nbgrader commands from anywhere on your system, but still have them operate on the same directory.
assignment_id corresponds to the unique name of an assignment.
The collect_step sub-directory corresponds to a step in the zip_collect workflow.

Workflow¶

The workflow for using nbgrader zip_collect is

You, as an instructor, place submitted files/archives in
```
{downloaded}/{assignment_id}/{archive_directory}
```
You run nbgrader zip_collect {assignment_id}, which will:

Extract these archive files - or copy non-archive files - to
{downloaded}/{assignment_id}/{extracted_directory}
Copy these extracted files to
{submitted_directory}/{student_id}/{assignment_id}/{notebook_id}.ipynb

At this point you can use nbgrader autograde to grade the files in the submitted directory (See Autograde assignments).

There are a few subtleties about how nbgrader zip_collect determines the correct student and notebook ids, which we’ll go through in the sections below.

Step 1: Download submission files or archives¶

The first step in the collect process is to extract files from any archive (zip) files - and copy any non-archive files - found in the following directory:

{downloaded}/{assignment_id}/{archive_directory}/

where:

The archive_directory contains the actual submission files or archives downloaded from your institution’s learning management system. nbgrader zip_collect assumes you have already created this directory and placed all downloaded submission files or archives in this directory.

For demo purposes we have already created the directories needed by the nbgrader zip_collect sub-command and placed the downloaded assignment submission files and archive (zip) files in there. For example we have one .zip file and one .ipynb file:

%%bash

ls -l downloaded/ps1/archive

total ##
-rw-r--r-- 1 nb_user nb_group [size] [date] [time] notebooks.zip
-rw------- 1 nb_user nb_group [size] [date] [time] ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb

But before we can run the nbgrader zip_collect sub-command we first need to specify a few config options:

%%file nbgrader_config.py

c = get_config()

# Only set for demo purposes so as to not mess up the other documentation
c.CourseDirectory.submitted_directory = 'submitted_zip'

# Only collect submitted notebooks with valid names
c.ZipCollectApp.strict = True

# Apply this regular expression to the extracted file filename (absolute path)
c.FileNameCollectorPlugin.named_regexp = (
    '.*_(?P<student_id>\w+)_attempt_'
    '(?P<timestamp>[0-9\-]+)_'
    '(?P<file_id>.*)'
)

Overwriting nbgrader_config.py

Setting the strict flag True skips any submitted notebooks with invalid names.

By default the nbgrader zip_collect sub-command uses the FileNameCollectorPlugin to collect files from the extracted_directory. This is done by sending each filename (absolute path) through to the FileNameCollectorPlugin, which in turn applies a named group regular expression (named_regexp) to the filename.

The FileNameCollectorPlugin returns None if the given file should be skipped or it returns an object that must contain the student_id and file_id data, and can optionally contain the timestamp, first_name, last_name, and email data.

Thus if using the default FileNameCollectorPlugin you must at least supply the student_id and file_id named groups. This plugin assumes all extracted files have the same filename or path structure similar to the downloaded notebook:

%%bash

ls -l downloaded/ps1/archive

total ##
-rw-r--r-- 1 nb_user nb_group [size] [date] [time] notebooks.zip
-rw------- 1 nb_user nb_group [size] [date] [time] ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb

Note

When collecting files in assignment sub-folders the file_id data must include the relative path to {assignment_id} and the filename in order to preserve the assignment directory structure.

If you wish to use a custom plugin for the ZipCollectApp see ZipCollect plugins for more information.

Before we extract the files, we also need to have run nbgrader generate_assignment:

%%bash

nbgrader generate_assignment "ps1" --IncludeHeaderFooter.header=source/header.ipynb --force

[GenerateAssignmentApp | WARNING] Removing existing assignment: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/ps1
[GenerateAssignmentApp | INFO] Copying [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/jupyter.png -> [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/./ps1/jupyter.png
[GenerateAssignmentApp | INFO] Updating/creating assignment 'ps1': {}
[GenerateAssignmentApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/problem1.ipynb
[GenerateAssignmentApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/ps1/problem1.ipynb
[GenerateAssignmentApp | INFO] Converting notebook [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/source/./ps1/problem2.ipynb
[GenerateAssignmentApp | INFO] Writing [size] bytes to [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/release/ps1/problem2.ipynb
[GenerateAssignmentApp | INFO] Setting destination file permissions to 644

Step 2: Extract, collect, and copy files¶

With the nbgrader_config.py file created we can now run the nbgrader zip_collect sub-command. This will:

Extract archive - or copy non-archive - files from the {archive_directory} into the following directory:
```
{downloaded}/{assignment_id}/{extracted_directory}/
```
Then collect and copy files from the extracted_directory above to the students submitted_directory:
```
{submitted_directory}/{student_id}/{assignment_id}/{notebook_id}.ipynb
```

For example:

%%bash

nbgrader zip_collect ps1 --force

[ZipCollectApp | INFO] Using file extractor: ExtractorPlugin
[ZipCollectApp | INFO] Using file collector: FileNameCollectorPlugin
[ZipCollectApp | WARNING] Directory not found. Creating: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/archive/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO] Extracting from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/archive/notebooks.zip
[ZipCollectApp | INFO]   Extracting to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks
[ZipCollectApp | INFO] Start collecting files...
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_myproblem1.ipynb
[ZipCollectApp | WARNING] Skipped notebook with invalid name 'myproblem1.ipynb'
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | WARNING] 4 files collected, 3 files skipped
[ZipCollectApp | INFO] Start transfering files...
[ZipCollectApp | WARNING] Directory not found. Creating: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem1.ipynb
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem2.ipynb
[ZipCollectApp | INFO] Creating timestamp: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/timestamp.txt
[ZipCollectApp | WARNING] Directory not found. Creating: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem2.ipynb
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem1.ipynb
[ZipCollectApp | INFO] Creating timestamp: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/timestamp.txt

After running the nbgrader zip_collect sub-command, the archive (zip) files were extracted - and the non-archive files were copied - to the extracted_directory:

%%bash

ls -l downloaded/ps1/extracted/
ls -l downloaded/ps1/extracted/notebooks/

total ##
drwxr-xr-x 1 nb_user nb_group [size] [date] [time] notebooks
-rw------- 1 nb_user nb_group [size] [date] [time] ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
total ##
-rw-rw-r-- 1 nb_user nb_group [size] [date] [time] ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
-rw-rw-r-- 1 nb_user nb_group [size] [date] [time] ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
-rw-rw-r-- 1 nb_user nb_group [size] [date] [time] ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
-rw-rw-r-- 1 nb_user nb_group [size] [date] [time] ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
-rw-rw-r-- 1 nb_user nb_group [size] [date] [time] ps1_hacker_attempt_2016-01-30-16-30-10_myproblem1.ipynb
-rw-rw-r-- 1 nb_user nb_group [size] [date] [time] ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb

By default archive files will be extracted into their own sub-directory in the extracted_directory and any archive files, within archive files, will also be extracted into their own sub-directory along the path. To change this default behavior you can write your own extractor plugin for zip_collect (see ZipCollect plugins).

These extracted files were then collected and copied into the students submitted_directory:

%%bash

ls -l submitted_zip

total ##
drwxr-xr-x 1 nb_user nb_group [size] [date] [time] bitdiddle
drwxr-xr-x 1 nb_user nb_group [size] [date] [time] hacker

%%bash

ls -l submitted_zip/hacker/ps1/

total ##
-rw------- 1 nb_user nb_group [size] [date] [time] problem1.ipynb
-rw-rw-r-- 1 nb_user nb_group [size] [date] [time] problem2.ipynb
-rw-r--r-- 1 nb_user nb_group [size] [date] [time] timestamp.txt

Custom plugins ¶

See also

ZipCollect plugins: Plugins for nbgrader zip_collect

Unfortunately, for the demo above, the timestamp strings from the filenames did not parse correctly:

%%bash

cat submitted_zip/hacker/ps1/timestamp.txt

2016-01-31 06:00:00

This is an issue with the underlying dateutils package used by nbgrader. But not to worry, we can easily create a custom collector plugin to correct the timestamp strings when the files are collected, for example:

%%file plugin.py

from nbgrader.plugins import FileNameCollectorPlugin


class CustomPlugin(FileNameCollectorPlugin):
    def collect(self, submission_file):
        info = super(CustomPlugin, self).collect(submission_file)
        if info is not None:
            info['timestamp'] = '{}-{}-{} {}:{}:{}'.format(
                *tuple(info['timestamp'].split('-'))
            )
        return info

Writing plugin.py

%%bash

# Use force flag to overwrite existing files
nbgrader zip_collect --force --collector=plugin.CustomPlugin ps1

[ZipCollectApp | INFO] Using file extractor: ExtractorPlugin
[ZipCollectApp | INFO] Using file collector: CustomPlugin
[ZipCollectApp | WARNING] Clearing existing files in [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/archive/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO] Extracting from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/archive/notebooks.zip
[ZipCollectApp | INFO]   Extracting to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks
[ZipCollectApp | INFO] Start collecting files...
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | WARNING] Skipped submission with no match information provided: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_jupyter.png
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_myproblem1.ipynb
[ZipCollectApp | WARNING] Skipped notebook with invalid name 'myproblem1.ipynb'
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO] Parsing file: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | WARNING] 4 files collected, 3 files skipped
[ZipCollectApp | INFO] Start transfering files...
[ZipCollectApp | WARNING] Clearing existing files in [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem1.ipynb
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_bitdiddle_attempt_2016-01-30-15-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/problem2.ipynb
[ZipCollectApp | INFO] Creating timestamp: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/bitdiddle/ps1/timestamp.txt
[ZipCollectApp | WARNING] Clearing existing files in [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/notebooks/ps1_hacker_attempt_2016-01-30-16-30-10_problem2.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem2.ipynb
[ZipCollectApp | INFO] Copying from: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/downloaded/ps1/extracted/ps1_hacker_attempt_2016-01-30-20-30-10_problem1.ipynb
[ZipCollectApp | INFO]   Copying to: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/problem1.ipynb
[ZipCollectApp | INFO] Creating timestamp: [NB_GRADER_ROOT]/nbgrader/docs/source/user_guide/submitted_zip/hacker/ps1/timestamp.txt

The --force flag is used this time to overwrite existing extracted and submitted files. Now if we check the timestamp we see it parsed correctly:

%%bash

cat submitted_zip/hacker/ps1/timestamp.txt

2016-01-30 20:30:10

Note that there should only ever be one instructor who runs the nbgrader zip_collect command (and there should probably only be one instructor – the same instructor – who runs nbgrader generate_assignment, nbgrader autograde and nbgrader formgrade as well). However this does not mean that only one instructor can do the grading, it just means that only one instructor manages the assignment files. Other instructors can still perform grading by accessing the formgrader URL.

Autograding resources¶

Most coding problems can be autograded. Problems that involve writing fruitful functions can be graded more easily than others. These types of problems can be graded by writing test functions that compare output values. Instructors should make sure that all edge cases are captured when creating test cases. Problems that require writing void functions are harder to autograde and may involve checking stdout, depending on the nature of the problem.

Here, we provide some best-practices and tips for writing autograder tests. If you have additional wisdom to add, please do open a PR (or even just an issue) on the nbgrader repository!

Tips for writing good test cases¶

Test each function/feature in isolation. If a problem contains many functions or parts, write cases that test each of these functions individually. Testing one function at a time makes it easier for you to track an error.

Organize test cases consistently. It can be helpful to arrange and group your test cases with comments.

Try to cover all edge cases. If you have a function that can take in a certain range of inputs, test the boundaries of this range. Test cases should also check for different lengths, different cases of strings, integers and floats, or different ranges when applicable.

Example¶

Problem: Write a function isAnagram() that takes 2 strings, and returns True if the two given strings are anagrams of each other. Your function should ignore cases, spaces, and all punctuation. So your function should identify “HeLLo!” and “hOlle” as anagrams.

Test cases:

from nose.tools import assert_equal

# standard True cases
assert_equal(isAnagram('hi', 'hi'), True)
assert_equal(isAnagram('pat', 'tap'), True)
assert_equal(isAnagram('left', 'felt'), True)

# ignore punctuation, spaces, and different cases (upper/lower)
assert_equal(isAnagram('hi', 'hi!'), True)
assert_equal(isAnagram('HI', 'hi'), True)
assert_equal(isAnagram('hi', 'HI'), True)
assert_equal(isAnagram('He llo', '?hello'), True)

# False cases
assert_equal(isAnagram('hi', 'h'), False)
assert_equal(isAnagram('apple', 'aple'), False)
assert_equal(isAnagram('aaaaaa', 'aaaa'), False)

Partially autograding, partially manually grading¶

When test cases are not enough to determine the correctness of a student’s solution, you can autograde them to make sure that there are no errors in the execution or the solution. You still need to manually look at the solutions to determine whether they are correct or not. This might be helpful if you want students to write a function for a problem using a specific implementation approach.

Example¶

Problem: Write a function sortList() that takes a list of numbers and returns a list sorted in descending order without using the built-in methods.

Test cases (but will still require instructors to check whether any built-in method is used):

from nose.tools import assert_equal
assert_equal(sortList([2, 3, 1]), [3, 2, 1])
assert_equal(sortList([3, 2, 1]), [3, 2, 1])
assert_equal(sortList([1, 2, 1, 2, 3, 1]), [3, 2, 2, 1, 1, 1])
assert_equal(sortList([-1, 0, 1]), [1, 0, -1])

Checking whether a specific function has been used¶

Sometimes, you may want to ensure that students are implementing their code in the way they have been asked to. For example, if you have your students write a function called mse` (to compute mean-squared-error) in the first part of the problem, and then want them to plot the MSE, you may ask them to be sure that they use the mse function in their code. How can you test for this?

In Python, you can be a bit clever and test for this by removing the mse function from the global namespace, running their code, and checking whether their code throws the appropriate error. If it doesn’t throw an error, that means they aren’t calling the mse function.

A test case that does this might look something like this:

# save a reference to the original function, then delete it from the
# global namespace
old_mse = mse
del mse

# try running the students' code
try:
    plot_mse()

# if an NameError is thrown, that means their function calls mse
except NameError:
    pass

# if no error is thrown, that means their function does not call mse
else:
    raise AssertionError("plot_mse does not call mse")

# restore the original function
finally:
    mse = old_mse
    del old_mse

Checking how functions were called or not called with mocking¶

In unit testing it is desirable to test your own code but not necessarily to include calls your code makes to external libraries. In particular calls that involve file systems and databases are typically replaced with so called mock objects that can record what arguments were passed in the call but which do not otherwise do any work.

We can use this in autograding in the following situations with the patch function from the mock library (since Python 3 a subpackage of unittest). The argument to patch is a string which represents where a certain function is used (rather than where it is defined). In the context of jupyter notebooks this argument will be ‘__main__.function_name’.

Example¶

Problem: verify that a function call results in the printing of a certain result

def foo()
    #...
    print('right result')

The test code for this can be written as

from unittest.mock import patch
with patch('__main__.print') as mock_print:
    foo()
mock_print.assert_called_once_with('right_result')

This test passes silently if the print is correct, but if the print is wrong:

def foo()
    #...
    print('wrong result')

an assertion error is raised with output of the form

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
...

AssertionError: Expected call: print('right result')
Actual call: print('wrong result')

Example¶

Problem: verify that the students have implemented a min function without using the built-in function. That is, you expect the student to write a solution like

def mymin(x, y):
    if x < y:
        return x
    else:
        return y

However, if the student has implemented

def mymin(x, y):
    return min(x, y)

the following test will capture this with an error

with patch('__main__.min') as mock_min:
    mymin(1, 2)

mock_min.assert_not_called()

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
...
AssertionError: Expected 'min' to not have been called. Called 1 times.

Grading plots¶

Programmatically grading plots is a painful experience because there are many ways that students can create the requested plot. In general, we recommend just grading plots by hand. However, it is possible to programmatically grade some simple types of plots (such as a scatter plot or bar plot). One such tool that facilitates grading matplotlib plot specifically is plotchecker.

Gotchas¶

In many ways, writing autograder tests is similar to writing unit tests. However, there are certain types of errors that students may make—especially if they are new to programming—that are not things you would typically test for when writing tests for your own code. Here are a list of some things we’ve come across that are good to be aware of when writing your own autograder tests.

For loops¶

For loops in Python are sometimes confusing to students who are new to programming, especially if they are sometimes using them in the context of indexing into lists/arrays and sometimes not. For example, I have seen students sometimes write a for loop like this:

for i in x:
    y[i] = f(x[i])

rather than:

for i in range(len(x)):
    y[i] = f(x[i])

In particular, if x in the above example contains integers, the code may not throw an error, and in certain cases, may even pass the tests if you are not looking out for this type of mistake!

Global variables in the notebook¶

Although the Jupyter notebook is a really excellent format for assignments, it does have some drawbacks. One drawback is the fact that variables defined earlier in the document—for example, in a piece of sample code—can be accessed later. This can pose problems if students accidentally use those variable names rather than the variable names given in the function definition.

As a toy example, let’s say that earlier in the notebook we have defined a variable called arr. Then, students are asked to write a function that multiplies all the variables in the array by two. You give them a function signature of f(x), and they write the following code:

def f(x):
    return arr * 2

Notice that their code uses arr rather than x. This can be a problem especially if you only test on one input—namely, arr—because that test case will pass! Thus, it is important to test students’ code on a variety of inputs in order to catch edge cases such as these, and that you use different variable names:

# both of these tests will pass no because the student hardcoded the use
# of the arr variable!
arr = np.array([1, 2, 3])
np.assert_array_equal(f(arr), np.array([2, 4, 6]))
arr = np.array([5, 7])
np.assert_array_equal(f(arr), np.array([10, 14]))

# this test will fail because it uses a new input AND a new variable name
arr2 = np.array([3, 2])
np.assert_array_equal(f(arr2), np.array([6, 4]))

One solution is to quietly reset the notebook environment using [%reset -f](https://ipython.readthedocs.io/en/stable/interactive/magics.html?highlight=magic#magic-reset) magic function. Note that this wipes out all variables, function definitions, and loaded libraries, but should leave downloaded libraries in local environment. This is useful if the test functions in a single notebook are completely independent of one another.

Alternatively, if you suspect a particular variable name be re-used in between tests, consider deleting these variables before running the tests.

Frequently asked questions ¶

Table of contents

Frequently asked questions

Can I use nbgrader for purely manually graded assignments (i.e., without autograding)?¶

Yes, absolutely! Mark all the cells where students write their answers as “Manually graded answer” cells and then during grading run nbgrader autograde and the formgrader as normal. If you don’t want to even execute the notebooks, you can pass the --no-execute flag to nbgrader autograde.

Can I assign my own cell ids rather than using the autogenerated ones?¶

Yes, by default nbgrader will assign your cells an id like cell-8dc50d93bc80e053, but you can alter this to be something more descriptive if you like. You must use unique names for each graded cell.

Can I hide the test cells in a nbgrader assignment?¶

Yes, as of version 0.5.0 of nbgrader you will be able to hide tests in “Autograder tests” cells (see “Autograder tests” cells with hidden tests).

How does nbgrader ensure that students do not change the tests?¶

Please see the documentation on “Read-only” cells.

Does nbgrader support parallel autograding of assignments?¶

Not yet, though it is on the todo list (see #174). PRs welcome!

Does nbgrader protect against infinite loops?¶

Yes. nbgrader will stop executing a cell after a certain period of time. This timeout is customizable through the ExecutePreprocessor.timeout configuration option. See Configuration options.

Does nbgrader protect against unsafe code?¶

Not directly, but you can use a kernel inside a Docker container to do secure grading. Please see Grading in a docker container for details.

How does nbgrader handle late assignments?¶

By default nbgrader won’t explicitly assign late penalties, but it will compute how late each submission is. If you wish to customize this default behavior see adding customization plugins.

For this to work, you must include a duedate for the assignment and then a timestamp.txt file in the folder for each submission with a single line containing a timestamp (e.g. 2015-02-02 14:58:23.948203 America/Los_Angeles). Then, when you run nbgrader autograde, nbgrader will record these timestamps into the database. You can access the timestamps through the API, like so:

from nbgrader.api import Gradebook
with Gradebook("sqlite:///gradebook.db") gb:
    assignment = gb.find_assignment("ps1")
    for submission in assignment.submissions:
        print("Submission from '{}' is {} seconds late".format(
            submission.student_id, submission.total_seconds_late))

Note that if you use the release_assignment/fetch_assignment/submit/collect commands (see Exchanging assignment files), the timestamp.txt files will be included automatically.

Do I have to use sqlite for the nbgrader database?¶

No, and in fact, if you have multiple people grading accessing the formgrader at the same time we strongly encourage you not to use sqlite because it is not threadsafe. Postgres is also supported, and anything else that works with SQLAlchemy is likely to work (e.g. MySQL), though only sqlite and Postgres have been tested. If you want to use another SQL-based database and find that it doesn’t work for some reason, please open an issue!

Does nbgrader work with non-Python kernels?¶

Yes, though it hasn’t been extensively tested with other kernels and it is likely there are some edge cases where things do not work quite right. One thing in particular that you will need to do is customize how the student version is produced – for example, you will need to change the delimiters for the solution regions to use the appropriate comment marks for your language.

If you run into any issues using nbgrader with other kernels, please open an issue!

How do I get out grade information from the database?¶

nbgrader offers a fairly rich API for interfacing with the database. Please see Getting information from the database for more details.

Can I use the “Assignment List” extension with multiple classes?¶

New in version 0.6.0.

Yes! To use the “Assignment List” extension with multiple courses, you will want to set the following config options in your students’ nbgrader_config.py files:

from nbgrader.auth JupyterHubAuthPlugin
c = get_config()
c.Exchange.path_includes_course = True
c.Authenticator.plugin_class = JupyterHubAuthPlugin

The first option (Exchange.path_includes_course) will tell the transfer apps (i.e. nbgrader fetch_assignment, nbgrader submit, and nbgrader list) to assume that the paths for assignments include the course name, such as ./course101/ps1 rather than just ./ps1 (which is the default). Then, when using the “Assignment List” extension, students will be able to switch between different classes.

The second option (Authenticator.plugin_class) tells nbgrader to ask JupyterHub which courses the student is enrolled in. It does require a bit more setup than simply editing the nbgrader_config.py file, however. Please see JupyterHub Authentication for details.

Is nbgrader compatible with Windows/Mac/Linux?¶

Linux and Mac¶

nbgrader is fully compatible with Linux and also with Mac (with the exception of JupyterHub integration, as JupyterHub does not run on Mac).

Windows¶

The core pieces of nbgrader will also work on Windows: the “Create Assignment” extension, nbgrader generate_assignment, nbgrader autograde, nbgrader formgrade, nbgrader generate_feedback, nbgrader validate, and nbgrader export.

However, the parts of nbgrader corresponding to file management (the “Assignment List” extension, nbgrader release_assignment, nbgrader fetch_assignment, nbgrader submit, nbgrader collect, nbgrader list, nbgrader release_feedback, nbgrader fetch_feedback) will not work under Windows.

What happens if I do some manual grading, and then rerun the autograder?¶

If you rerun the autograder, nbgrader will never overwrite any manual grades or comments that you have added, and manual grades always take precedence over autogrades.

However, if you have given a manual grade, then rerun the autograder, and the autograder produces a grade as well, then it will mark that problem as “needing manual grade”. This functionality is primarily to aid you in grading in the scenarios where you want to grade a newer version of the student’s submission—for example, if you gave them a chance to revise it. In this hypothetical scenario, a student might have not completed a problem, leading you to originally assign it a low partial credit score. But then they turn in a newer version, which you run through the autograder and which attains full credit. Since the manual grade always takes precedence over the autograde, the student would still receive the low score unless you updated your grade: hence the motivation for marking it as needing to be manually graded (again).

Do students have to install anything on their own computers to use nbgrader?¶

No, nbgrader only needs to be installed for the instructor. However, students may optionally install the Validate extension to verify that their submission passes all the test cases.

What should I do if validation or grading of a notebook fails with a “Timeout waiting for execute reply” error?¶

This occurs because the validator or autograder is taking too long to validate or autograde your notebook. This can be fixed by adding the following line to nbgrader_config.py:

# increase timeout to 60 seconds
c.ExecutePreprocessor.timeout = 60

Can tests be only temporarily hidden, so that students can reveal them?¶

No, the tests are either present in the student version of the notebook or they are not. However, there exist extensions such as https://github.com/kirbs-/hide_code which can assist in hiding code cells.

Can I modify the notebook behavior during autograding or validation?¶

Yes, when running the autograder or validator, nbgrader sets the NBGRADER_EXECUTION environment variable, either to 'autograde' or 'validate'. You can check whether this variable is set and then modify the logic in the notebook accordingly. For example, you could use this to skip executing some cells during autograding only.

Advanced topics ¶

This file covers some more advanced use cases of nbgrader.

Table of contents

Advanced topics

Running nbgrader with JupyterHub ¶

Please see Using nbgrader with JupyterHub.

Advanced “Assignment List” installation ¶

See also

Installation

General installation instructions.

Exchanging assignment files

Details on fetching and submitting assignments using the “Assignment List”: plugin.

Warning

The “Assignment List” extension is not currently compatible with multiple courses on the same server: it will only work if there is a single course on the server. This is a known issue (see #544). PRs welcome!

This section covers some further and configuration scenarios that often occur with the assignment list extension.

In previous versions of nbgrader, a special process had to be used to enable this extension for all users on a multi-user system. As described in the main Installation documentation this is no longer required.

If you know you have released an assignment but still don’t see it in the list of assignments, check the output of the notebook server to see if there are any errors. If you do in fact see an error, try running the command manually on the command line from the directory where the notebook server is running. For example:

$ nbgrader list
[ListApp | ERROR] Unwritable directory, please contact your instructor: /srv/nbgrader/exchange

This error that the exchange directory isn’t writable is an easy mistake to make, but also relatively easy to fix. If the exchange directory is at /srv/nbgrader/exchange, then make sure you have run:

chmod +rw /srv/nbgrader/exchange

Getting information from the database ¶

nbgrader offers a fairly rich API for interfacing with the database. The API should allow you to access pretty much anything you want, though if you find something that can’t be accessed through the API please open an issue!

In this example, we’ll go through how to create a CSV file of grades for each student and assignment using nbgrader and pandas.

New in version 0.4.0: nbgrader now comes with CSV export functionality out-of-the box using the nbgrader export command. However, this example is still kept for reference as it may be useful for defining your own exporter.

import pandas as pd
from nbgrader.api import Gradebook, MissingEntry

# Create the connection to the database
with Gradebook('sqlite:///gradebook.db') as gb:

    grades = []

    # Loop over each assignment in the database
    for assignment in gb.assignments:

        # Loop over each student in the database
        for student in gb.students:

            # Create a dictionary that will store information about this student's
            # submitted assignment
            score = {}
            score['max_score'] = assignment.max_score
            score['student'] = student.id
            score['assignment'] = assignment.name

            # Try to find the submission in the database. If it doesn't exist, the
            # `MissingEntry` exception will be raised, which means the student
            # didn't submit anything, so we assign them a score of zero.
            try:
                submission = gb.find_submission(assignment.name, student.id)
            except MissingEntry:
                score['score'] = 0.0
            else:
                score['score'] = submission.score

            grades.append(score)

    # Create a pandas dataframe with our grade information, and save it to disk
    grades = pd.DataFrame(grades).set_index(['student', 'assignment']).sortlevel()
    grades.to_csv('grades.csv')

    # Print out what the grades look like
    with open('grades.csv', 'r') as fh:
        print(fh.read())

After running the above code, you should see that grades.csv contains something that looks like:

student,assignment,max_score,score
bitdiddle,ps1,9.0,1.5
hacker,ps1,9.0,3.0

Using nbgrader preprocessors ¶

Several of the nbgrader preprocessors can be used with nbconvert without actually relying on the rest of the nbgrader machinery. In particular, the following preprocessors can be applied to other nbconvert workflows:

ClearOutput – clears outputs of all cells
ClearSolutions – removes solutions between the solution delimeters (see “Autograded answer” cells).
HeaderFooter – concatenates notebooks together, prepending a “header” notebook and/or appending a “footer” notebook to another notebook.
LimitOutput – limits the amount of output any given cell can have. If a cell has too many lines of outputs, they will be truncated.

Using these preprocessors in your own nbconvert workflow is relatively straightforward. In your nbconvert_config.py file, you would add, for example:

c.Exporter.preprocessors = ['nbgrader.preprocessors.ClearSolutions']

See also the nbconvert docs on custom preprocessors.

Calling nbgrader apps from Python ¶

New in version 0.5.0: Much of nbgrader’s high level functionality can now be accessed through an official Python API.

Grading in a docker container ¶

For security reasons, it may be advantageous to do the grading with a kernel running in isolation, e.g. in a docker container. We will assume that docker is already installed and an appropriate image has been downloaded. Otherwise, refer to the docker documentation for information on how to install and run docker.

A convenient way to switch to a kernel running in a docker container is provided by envkernel which serves a double purpose. In a first step, it is writing a new kernelspec file. Later it ensures that the docker container is run and the kernel started.

Presently, envkernel is only available from its Github repository and can be installed directly from there into a virtual environment

pip install https://github.com/NordicHPC/envkernel/archive/master.zip

As an alternative, the script envkernel.py can be put in a different location, e.g. /opt/envkernel, as long as it is accessible there also later during grading.

Now, a new kernel can be installed by means of

./envkernel.py docker --name=NAME --display-name=DNAME DOCKER-IMAGE

Here, NAME should be replaced by the name to be given to the kernel. After installation of the kernel, it will be displayed in the list of kernels when executing jupyter kernelspec list. DNAME should be replaced by the name under which the kernel shall be known in the Jupyter notebook GUI. After installation of the kernel, this name will be listed as a possible kernel when starting a new notebook. Finally, DOCKER-IMAGE should be replaced by the name of the docker image in which the kernel is to be run, e.g. python:3, continuumio/anaconda3, or some other suitable image.

The command given above will install the kernel in the system-wide location for Jupyter data files. If installation in the corresponding user directory is desired, the option --user should be added before the name of the docker image. By default, envkernel will install a Python kernel. For the installation of other kernels, see the README of envkernel.

In order to run the grading process with the new kernel, one can specify its name in nbgrader_config.py

c.ExecutePreprocessor.kernel_name = NAME

where NAME should be replaced by the name chosen when running the envkernel script. Alternatively, the name can be specified when running nbgrader from the command line

nbgrader autograde --ExecutePreprocessor.kernel_name=NAME ASSIGNMENT_NAME

In addition to docker, envkernel also supports singularity as a containerization system. For details on using envkernel with singularity, see the README of envkernel.

API library documentation¶

High-Level API¶

New in version 0.5.0.

This API is a high-level api that provides access to nbgrader’s core functionality, for example assigning, releasing, collecting, and autograding assignments. For example:

from nbgrader.apps import NbGraderAPI
from traitlets.config import Config

# create a custom config object to specify options for nbgrader
config = Config()
config.CourseDirectory.course_id = "course101"

api = NbGraderAPI(config=config)

# assuming source/ps1 exists
api.generate_assignment("ps1")

For details on how to configure the API, see Configuration options.

class nbgrader.apps.api.NbGraderAPI(**kwargs)[source]¶

A high-level API for using nbgrader.

__init__(coursedir=None, authenticator=None, exchange=None, **kwargs)[source]¶

Initialize the API.

Parameters

coursedir (nbgrader.coursedir.CourseDirectory) – (Optional) A course directory object.
authenticator (:class:~`nbgrader.auth.BaseAuthenticator`) – (Optional) An authenticator instance for communicating with an external database.
exchange (:class:~`nbgrader.exchange.ExchangeFactory`) – (Optional) A factory for creating the exchange classes used for distributing assignments and feedback.
kwargs – Additional keyword arguments (e.g. parent, config)

gradebook¶

An instance of nbgrader.api.Gradebook.

Note that each time this property is accessed, a new gradebook is created. The user is responsible for destroying the gradebook through close().

get_source_assignments()[source]¶

Get the names of all assignments in the source directory.

Returns: assignments – A set of assignment names
Return type: set

get_released_assignments()[source]¶

Get the names of all assignments that have been released to the exchange directory. If the course id is blank, this returns an empty set.

Returns: assignments – A set of assignment names
Return type: set

get_submitted_students(assignment_id)[source]¶

Get the ids of students that have submitted a given assignment (determined by whether or not a submission exists in the submitted directory).

Parameters: assignment_id (string) – The name of the assignment. May be * to select for all assignments.
Returns: students – A set of student ids
Return type: set

get_submitted_timestamp(assignment_id, student_id)[source]¶

Gets the timestamp of a submitted assignment.

Parameters

assignment_id (string) – The assignment name
student_id (string) – The student id

Returns

timestamp – The timestamp of the submission, or None if the timestamp does not exist

Return type

datetime.datetime or None

get_autograded_students(assignment_id)[source]¶

Get the ids of students whose submission for a given assignment has been autograded. This is determined based on satisfying all of the following criteria:

There is a directory present in the autograded directory.
The submission is present in the database.
The timestamp of the autograded submission is the same as the timestamp of the original submission (in the submitted directory).

Returns: students – A set of student ids
Return type: set

get_assignment(assignment_id, released=None)[source]¶

Get information about an assignment given its name.

Parameters

assignment_id (string) – The name of the assignment
released (list) – (Optional) A set of names of released assignments, obtained via self.get_released_assignments().

Returns

assignment – A dictionary containing information about the assignment

Return type

dict

get_assignments()[source]¶

Get a list of information about all assignments.

Returns: assignments – A list of dictionaries containing information about each assignment
Return type: list

get_notebooks(assignment_id)[source]¶

Get a list of notebooks in an assignment.

Parameters: assignment_id (string) – The name of the assignment
Returns: notebooks – A list of dictionaries containing information about each notebook
Return type: list

get_submission(assignment_id, student_id, ungraded=None, students=None)[source]¶

Get information about a student’s submission of an assignment.

Parameters

assignment_id (string) – The name of the assignment
student_id (string) – The student’s id
ungraded (set) – (Optional) A set of student ids corresponding to students whose submissions have not yet been autograded.
students (dict) – (Optional) A dictionary of dictionaries, keyed by student id, containing information about students.

Returns

submission – A dictionary containing information about the submission

Return type

dict

get_submissions(assignment_id)[source]¶

Get a list of submissions of an assignment. Each submission corresponds to a student.

Parameters: assignment_id (string) – The name of the assignment
Returns: notebooks – A list of dictionaries containing information about each submission
Return type: list

get_notebook_submission_indices(assignment_id, notebook_id)[source]¶

Get a dictionary mapping unique submission ids to indices of the submissions relative to the full list of submissions.

Parameters

assignment_id (string) – The name of the assignment
notebook_id (string) – The name of the notebook

Returns

indices – A dictionary mapping submission ids to the index of each submission

Return type

dict

get_notebook_submissions(assignment_id, notebook_id)[source]¶

Get a list of submissions for a particular notebook in an assignment.

Parameters

assignment_id (string) – The name of the assignment
notebook_id (string) – The name of the notebook

Returns

submissions – A list of dictionaries containing information about each submission.

Return type

list

get_student(student_id, submitted=None)[source]¶

Get a dictionary containing information about the given student.

Parameters

student_id (string) – The unique id of the student
submitted (set) – (Optional) A set of unique ids of students who have submitted an assignment

Returns

student – A dictionary containing information about the student, or None if the student does not exist

Return type

dictionary

get_students()[source]¶

Get a list containing information about all the students in class.

Returns: students – A list of dictionaries containing information about all the students
Return type: list

get_student_submissions(student_id)[source]¶

Get information about all submissions from a particular student.

Parameters: student_id (string) – The unique id of the student
Returns: submissions – A list of dictionaries containing information about all the student’s submissions
Return type: list

get_student_notebook_submissions(student_id, assignment_id)[source]¶

Gets information about all notebooks within a submitted assignment.

Parameters

student_id (string) – The unique id of the student
assignment_id (string) – The name of the assignment

Returns

submissions – A list of dictionaries containing information about the submissions

Return type

list

assign(*args, **kwargs)[source]¶: Deprecated, please use generate_assignment instead.

release(*args, **kwargs)[source]¶: Deprecated, please use release_assignment instead.

unrelease(assignment_id)[source]¶

Run nbgrader list --remove for a particular assignment.

Parameters

assignment_id (string) – The name of the assignment

Returns

result – A dictionary with the following keys (error and log may or may not be present):

success (bool): whether or not the operation completed successfully
error (string): formatted traceback
log (string): captured log output

Return type

dict

collect(assignment_id, update=True)[source]¶

Run nbgrader collect for a particular assignment.

Parameters

assignment_id (string) – The name of the assignment
update (bool) – Whether to update already-collected assignments with newer submissions, if they exist

Returns

result – A dictionary with the following keys (error and log may or may not be present):

success (bool): whether or not the operation completed successfully
error (string): formatted traceback
log (string): captured log output

Return type

dict

autograde(assignment_id, student_id, force=True, create=True)[source]¶

Run nbgrader autograde for a particular assignment and student.

Parameters

assignment_id (string) – The name of the assignment
student_id (string) – The unique id of the student
force (bool) – Whether to autograde the submission, even if it’s already been autograded
create (bool) – Whether to create students in the database if they don’t already exist

Returns

result – A dictionary with the following keys (error and log may or may not be present):

success (bool): whether or not the operation completed successfully
error (string): formatted traceback
log (string): captured log output

Return type

dict

generate_feedback(assignment_id, student_id=None, force=True)[source]¶

Run nbgrader generate_feedback for a particular assignment and student.

Parameters

assignment_id (string) – The name of the assignment
student_id (string) – The name of the student (optional). If not provided, then generate feedback from autograded submissions.
force (bool) – Whether to force generating feedback, even if it already exists.

Returns

result – A dictionary with the following keys (error and log may or may not be present):

success (bool): whether or not the operation completed successfully
error (string): formatted traceback
log (string): captured log output

Return type

dict

release_feedback(assignment_id, student_id=None)[source]¶

Run nbgrader release_feedback for a particular assignment/student.

Parameters

assignment_id (string) – The name of the assignment
assignment_id – The name of the student (optional). If not provided, then release all generated feedback.

Returns

result – A dictionary with the following keys (error and log may or may not be present):

success (bool): whether or not the operation completed successfully
error (string): formatted traceback
log (string): captured log output

Return type

dict

fetch_feedback(assignment_id, student_id)[source]¶

Run nbgrader fetch_feedback for a particular assignment/student.

Parameters

assignment_id (string) – The name of the assignment
student_id (string) – The name of the student.

Returns

result – A dictionary with the following keys (error and log may or may not be present):

success (bool): whether or not the operation completed successfully
error (string): formatted traceback
log (string): captured log output
value (list of dict): all submitted assignments

Return type

dict

Database models¶

In general, these database models should never be modified by hand. You should only ever modify them using a Gradebook object, so that changes are properly persisted to the database, and so that the models don’t end up in an inconsistent state. However, some methods of the Gradebook object return database model objects, so those models and their attributes are documented here for reference.

class nbgrader.api.Student(**kwargs)[source]¶

Database representation of a student.

id¶: Unique id of the student. This could be a student ID, a username, an email address, etc., so long as it is unique.

first_name¶: (Optional) The first name of the student

last_name¶: (Optional) The last name of the student

email¶: (Optional) The student’s email address, if the id does not correspond to an email address

score¶: The overall score of the student across all assignments, computed automatically from the score of each submitted assignment.

max_score¶: The maximum possible score the student could achieve across all assignments, computed automatically from the max_score of each assignment.

lms_user_id¶: The LMS user ID, this is mainly for identifying students in your LMS system and was added so teachers and TA’s can easily send grades to a LMS such as Canvas and Blackboard.

submissions¶: A collection of assignments submitted by the student, represented as SubmittedAssignment objects

to_dict()[source]¶: Convert the student object to a JSON-friendly dictionary representation.

Master version of an assignment¶

class nbgrader.api.Assignment(name, duedate=None, course_id='default_course', **kwargs)[source]¶

Database representation of the master/source version of an assignment.

id¶: Unique id of the assignment (automatically generated)

name¶: Unique human-readable name for the assignment, such as “Problem Set 1”

duedate¶: (Optional) Duedate for the assignment in datetime format, with UTC timezone

course_id¶: The course for this assignment

course¶

notebooks¶: A collection of notebooks contained in this assignment, represented by Notebook objects

submissions¶: A collection of submissions of this assignment, represented by SubmittedAssignment objects.

num_submissions¶: The number of submissions of this assignment

max_score¶: Maximum score achievable on this assignment, automatically calculated from the max_score of each notebook

max_code_score¶: Maximum coding score achievable on this assignment, automatically calculated from the max_code_score of each notebook

max_written_score¶: Maximum written score achievable on this assignment, automatically calculated from the max_written_score of each notebook

to_dict()[source]¶: Convert the assignment object to a JSON-friendly dictionary representation.

class nbgrader.api.Notebook(**kwargs)[source]¶

Database representation of the master/source version of a notebook.

id¶: Unique id of the notebook (automatically generated)

name¶: Unique human-readable name for the notebook, such as “Problem 1”. Note the uniqueness is only constrained within assignments (e.g. it is ok for two different assignments to both have notebooks called “Problem 1”, but the same assignment cannot have two notebooks with the same name).

assignment¶: The Assignment object that this notebook is a part of

assignment_id¶: Unique id of assignment

kernelspec¶: The json string representation of the kernelspec for this notebook

grade_cells¶

solution_cells¶

task_cells¶

source_cells¶: A collection of source cells contained within this notebook, represented by SourceCell objects

submissions¶: A collection of submitted versions of this notebook, represented by SubmittedNotebook objects

num_submissions¶: The number of submissions of this notebook

max_score¶: Maximum score achievable on this notebook, automatically calculated from the max_score of each grade cell

max_code_score¶: Maximum coding score achievable on this notebook, automatically calculated from the max_score and cell_type of each grade cell

max_written_score¶: Maximum written score achievable on this notebook, automatically calculated from the max_score and cell_type of each grade cell

needs_manual_grade¶: Whether there are any submitted versions of this notebook that need to be manually graded, automatically determined from the needs_manual_grade attribute of each submitted notebook

to_dict()[source]¶: Convert the notebook object to a JSON-friendly dictionary representation.

class nbgrader.api.GradedMixin[source]¶

Mixin class providing the reference to a grade and the data members relevant for graded cells.

max_score = Column(None, Float(), table=None, nullable=False)¶

cell_type = Column(None, Enum('code', 'markdown', name='grade_cell_type'), table=None, nullable=False)¶

class nbgrader.api.BaseCell(**kwargs)[source]¶

Database representation of a cell. It is meant as a base class for cells where additional behavior is added through mixin classes.

id¶: Unique id of the grade cell (automatically generated)

name¶: Unique human-readable name of the cell. This need only be unique within the notebook, not across notebooks.

notebook¶: The notebook that this cell is contained within, represented by a Notebook object

notebook_id¶: Unique id of the notebook

assignment¶

class nbgrader.api.GradeCell(**kwargs)[source]¶

Bases: BaseCell, GradedMixin

Database representation of the master/source version of a grade cell.

id¶: Unique id of the cell (automatically generated from BaseCell)

grades¶: A collection of grades associated with this cell, represented by Grade objects

to_dict()[source]¶: Convert the grade cell object to a JSON-friendly dictionary representation. Note that this includes keys for notebook and assignment which correspond to the names of the notebook and assignment, not the objects themselves.

class nbgrader.api.SolutionCell(**kwargs)[source]¶

id¶: Unique id of the cell (automatically generated from BaseCell)

comments¶: A collection of comments associated with this cell, represented by Comment objects

to_dict()[source]¶: Convert the solution cell object to a JSON-friendly dictionary representation. Note that this includes keys for notebook and assignment which correspond to the names of the notebook and assignment, not the objects themselves.

class nbgrader.api.TaskCell(**kwargs)[source]¶

Bases: BaseCell, GradedMixin

Database representation of a task cell.

id¶: Unique id of the cell (automatically generated from BaseCell)

grades¶: A collection of grades associated with this cell, represented by Grade objects

comments¶: A collection of comments associated with this cell, represented by Comment objects

to_dict()[source]¶: Convert the task cell object to a JSON-friendly dictionary representation. Note that this includes keys for notebook and assignment which correspond to the names of the notebook and assignment, not the objects themselves.

class nbgrader.api.SourceCell(**kwargs)[source]¶

id¶: Unique id of the source cell (automatically generated)

name¶: Unique human-readable name of the source cell. This need only be unique within the notebook, not across notebooks.

cell_type¶: The cell type, either “code” or “markdown”

locked¶: Whether the cell is locked (e.g. the source saved in the database should be used to overwrite the source of students’ cells)

source¶: The source code or text of the cell

checksum¶: A checksum of the cell contents. This should usually be computed using nbgrader.utils.compute_checksum()

notebook¶: The Notebook that this source cell is contained in

notebook_id¶: Unique id of the notebook

assignment = ObjectAssociationProxyInstance(AssociationProxy('notebook', 'assignment'))¶: The assignment that this cell is contained within, represented by a Assignment object

to_dict()[source]¶: Convert the source cell object to a JSON-friendly dictionary representation. Note that this includes keys for notebook and assignment which correspond to the names of the notebook and assignment, not the objects themselves.

Submitted assignments¶

class nbgrader.api.SubmittedAssignment(**kwargs)[source]¶

Database representation of an assignment submitted by a student.

id¶: Unique id of the submitted assignment (automatically generated)

name = ColumnAssociationProxyInstance(AssociationProxy('assignment', 'name'))¶: Name of the assignment, inherited from Assignment

assignment¶: The master version of this assignment, represented by a Assignment object

assignment_id¶: Unique id of assignment

student¶: The student who submitted this assignment, represented by a Student object

student_id¶: Unique id of student

timestamp¶: (Optional) The date and time that the assignment was submitted, in date time format with a UTC timezone

extension¶: (Optional) An extension given to the student for this assignment, in time delta format

duedate¶

The duedate of this student’s assignment, which includes any extension given, if applicable, and which is just the regular assignment duedate otherwise.

Return type: datetime

total_seconds_late¶

The number of seconds that this assignment was turned in past the duedate (including extensions, if any). If the assignment was turned in before the deadline, this value will just be zero.

Return type: float

notebooks¶: A collection of notebooks contained within this submitted assignment, represented by SubmittedNotebook objects

score¶: The score assigned to this assignment, automatically calculated from the score of each notebook within this submitted assignment.

max_score¶: The maximum possible score of this assignment, inherited from Assignment

code_score¶: The code score assigned to this assignment, automatically calculated from the code_score of each notebook within this submitted assignment.

max_code_score¶: The maximum possible code score of this assignment, inherited from Assignment

written_score¶: The written score assigned to this assignment, automatically calculated from the written_score of each notebook within this submitted assignment.

max_written_score¶: The maximum possible written score of this assignment, inherited from Assignment

needs_manual_grade¶: Whether this assignment has parts that need to be manually graded, automatically determined from the needs_manual_grade attribute of each notebook.

late_submission_penalty¶: The penalty (>= 0) given for submitting the assignment late. Automatically determined from the late_submission_penalty attribute of each notebook.

to_dict()[source]¶: Convert the submitted assignment object to a JSON-friendly dictionary representation. Note that this includes a student key which is the unique id of the student, not the object itself.

class nbgrader.api.SubmittedNotebook(**kwargs)[source]¶

Database representation of a notebook submitted by a student.

id¶: Unique id of the submitted notebook (automatically generated)

name = ColumnAssociationProxyInstance(AssociationProxy('notebook', 'name'))¶: Name of the notebook, inherited from Notebook

assignment¶: The submitted assignment this notebook is a part of, represented by a SubmittedAssignment object

assignment_id¶: Unique id of assignment

notebook¶: The master version of this notebook, represented by a Notebook object

notebook_id¶: Unique id of notebook

grades¶: Collection of associated with this submitted notebook, represented by Grade objects

comments¶: Collection of comments associated with this submitted notebook, represented by Comment objects

student = ObjectAssociationProxyInstance(AssociationProxy('assignment', 'student'))¶: The student who submitted this notebook, represented by a Student object

flagged¶: Whether this assignment has been flagged by a human grader

score¶: The score assigned to this notebook, automatically calculated from the score of each grade cell within this submitted notebook.

max_score¶: The maximum possible score of this notebook, inherited from Notebook

code_score¶: The code score assigned to this notebook, automatically calculated from the score and cell_type of each grade within this submitted notebook.

max_code_score¶: The maximum possible code score of this notebook, inherited from Notebook

written_score¶: The written score assigned to this notebook, automatically calculated from the score and cell_type of each grade within this submitted notebook.

max_written_score¶: The maximum possible written score of this notebook, inherited from Notebook

needs_manual_grade¶: Whether this notebook has parts that need to be manually graded, automatically determined from the needs_manual_grade attribute of each grade.

failed_tests¶: Whether this notebook contains autograder tests that failed to pass, automatically determined from the failed_tests attribute of each grade.

late_submission_penalty¶: The penalty (>= 0) given for submitting the assignment late. Updated by the LateSubmissionPlugin.

class nbgrader.api.Grade(**kwargs)[source]¶

Database representation of a grade assigned to the submitted version of a grade cell.

id¶: Unique id of the grade (automatically generated)

name = ColumnAssociationProxyInstance(AssociationProxy('cell', 'name'))¶: Unique name of the grade cell, inherited from GradeCell

assignment = ObjectAssociationProxyInstance(AssociationProxy('notebook', 'assignment'))¶: The submitted assignment that this grade is contained in, represented by a SubmittedAssignment object

notebook¶: The submitted notebook that this grade is assigned to, represented by a SubmittedNotebook object

notebook_id¶: Unique id of notebook

cell¶: The master version of the cell this grade is assigned to, represented by a GradeCell object.

cell_id¶: Unique id of cell

cell_type¶: The type of cell this grade corresponds to, inherited from GradeCell

student = ObjectAssociationProxyInstance(AssociationProxy('notebook', 'student'))¶: The student who this grade is assigned to, represented by a Student object

auto_score¶: Score assigned by the autograder

manual_score¶: Score assigned by a human grader

extra_credit¶: Extra credit assigned by a human grader

score¶: The overall score, computed automatically from the auto_score and manual_score values. If neither are set, the score is zero. If both are set, then the manual score takes precedence. If only one is set, then that value is used for the score.

max_score¶

needs_manual_grade¶: Whether a score needs to be assigned manually. This is True by default.

failed_tests¶: Whether the autograded score is a result of failed autograder tests. This is True if the autograder score is zero and the cell type is “code”, and otherwise False.

to_dict()[source]¶: Convert the grade object to a JSON-friendly dictionary representation. Note that this includes keys for notebook and assignment which correspond to the name of the notebook and assignment, not the actual objects. It also includes a key for student which corresponds to the unique id of the student, not the actual student object.

class nbgrader.api.Comment(**kwargs)[source]¶

Database representation of a comment on a cell in a submitted notebook.

id¶: Unique id of the comment (automatically generated)

name = ColumnAssociationProxyInstance(AssociationProxy('cell', 'name'))¶: Unique name of the solution cell, inherited from SolutionCell

assignment = ObjectAssociationProxyInstance(AssociationProxy('notebook', 'assignment'))¶: The submitted assignment that this comment is contained in, represented by a SubmittedAssignment object

notebook¶: The submitted notebook that this comment is assigned to, represented by a SubmittedNotebook object

notebook_id¶: Unique id of notebook

cell¶: The master version of the cell this comment is assigned to, represented by a SolutionCell object.

cell_id¶: Unique id of cell

student = ObjectAssociationProxyInstance(AssociationProxy('notebook', 'student'))¶: The student who this comment is assigned to, represented by a Student object

auto_comment¶: A comment which is automatically assigned by the autograder

manual_comment¶: A comment which is assigned manually

comment¶: The overall comment, computed automatically from the auto_comment and manual_comment values. If neither are set, the comment is None. If both are set, then the manual comment takes precedence. If only one is set, then that value is used for the comment.

to_dict()[source]¶: Convert the comment object to a JSON-friendly dictionary representation. Note that this includes keys for notebook and assignment which correspond to the name of the notebook and assignment, not the actual objects. It also includes a key for student which corresponds to the unique id of the student, not the actual student object.

class nbgrader.api.Course(**kwargs)[source]¶

Table to store the courses

id¶

assignments¶

Gradebook¶

class nbgrader.api.Gradebook(db_url, course_id='default_course', authenticator=None)[source]¶

The gradebook object to interface with the database holding nbgrader grades.

__init__(db_url, course_id='default_course', authenticator=None)[source]¶

Initialize the connection to the database.

Parameters

db_url (str) – The URL to the database, e.g. sqlite:///grades.db
course_id (str) – identifier of the course necessary for supporting multiple classes default course_id is ‘’ to be consistent with :class:~`nbgrader.apps.api.NbGraderAPI`
authenticator (Optional[Authenticator]) – An authenticator instance for communicating with an external database.

close()[source]¶

Close the connection to the database.

It is important to call this method after you are done using the gradebook. In particular, if you create multiple instances of the gradebook without closing them, you may run into errors where there are too many open connections to the database.

check_course(course_id='default_course', **kwargs)[source]¶

Set the course id

Parameters

course_id (string) – The unique id of the course
**kwargs (dict) – other keyword arguments to the Course object

Returns

course

Return type

Course

students¶

A list of all students in the database.

Return type: List[Student]

add_student(student_id, **kwargs)[source]¶

Add a new student to the database.

Parameters

student_id (str) – The unique id of the student
**kwargs – other keyword arguments to the Student object

Return type

student

find_student(student_id)[source]¶

Find a student.

Parameters: student_id (str) – The unique id of the student
Return type: student

update_or_create_student(student_id, **kwargs)[source]¶

Update an existing student, or create it if it doesn’t exist.

Parameters

student_id (str) – The unique id of the student
**kwargs – additional keyword arguments for the Student object

Return type

student

remove_student(student_id)[source]¶

Deletes an existing student from the gradebook, including any submissions the might be associated with that student.

Parameters: student_id (string) – The unique id of the student

assignments¶

A list of all assignments in the gradebook.

Return type: List[Assignment]

add_assignment(name, **kwargs)[source]¶

Add a new assignment to the gradebook.

Parameters

name (str) – the unique name of the new assignment
**kwargs – additional keyword arguments for the Assignment object

Return type

assignment

find_assignment(name)[source]¶

Find an assignment in the gradebook.

Parameters: name (str) – the unique name of the assignment
Return type: assignment

update_or_create_assignment(name, **kwargs)[source]¶

Update an existing assignment, or create it if it doesn’t exist.

Parameters

name (str) – the name of the assignment
**kwargs – additional keyword arguments for the Assignment object

Return type

assignment

remove_assignment(name)[source]¶

Deletes an existing assignment from the gradebook, including any submissions the might be associated with that assignment.

Parameters: name (string) – the name of the assignment to delete

add_notebook(name, assignment, **kwargs)[source]¶

Add a new notebook to an assignment.

Parameters

name (str) – the name of the new notebook
assignment (str) – the name of an existing assignment
**kwargs – additional keyword arguments for the Notebook object

Return type

notebook

find_notebook(name, assignment)[source]¶

Find a particular notebook in an assignment.

Parameters

name (str) – the name of the notebook
assignment (str) – the name of the assignment

Return type

notebook

update_or_create_notebook(name, assignment, **kwargs)[source]¶

Update an existing notebook, or create it if it doesn’t exist.

Parameters

name (string) – the name of the notebook
assignment (string) – the name of the assignment
**kwargs – additional keyword arguments for the Notebook object

Returns

notebook

Return type

Notebook

remove_notebook(name, assignment)[source]¶

Deletes an existing notebook from the gradebook, including any submissions the might be associated with that notebook.

Parameters

name (string) – the name of the notebook to delete
assignment (string) – the name of an existing assignment

add_grade_cell(name, notebook, assignment, **kwargs)[source]¶

Add a new grade cell to an existing notebook of an existing assignment.

Parameters

name (str) – the name of the new grade cell
notebook (str) – the name of an existing notebook
assignment (str) – the name of an existing assignment
**kwargs – additional keyword arguments for GradeCell

Return type

grade_cell

find_grade_cell(name, notebook, assignment)[source]¶

Find a grade cell in a particular notebook of an assignment.

Parameters

name (str) – the name of the grade cell
notebook (str) – the name of the notebook
assignment (str) – the name of the assignment

Return type

grade_cell

update_or_create_grade_cell(name, notebook, assignment, **kwargs)[source]¶

Update an existing grade cell in a notebook of an assignment, or create the grade cell if it does not exist.

Parameters

name (str) – the name of the grade cell
notebook (str) – the name of the notebook
assignment (str) – the name of the assignment
**kwargs – additional keyword arguments for GradeCell

Return type

grade_cell

add_solution_cell(name, notebook, assignment, **kwargs)[source]¶

Add a new solution cell to an existing notebook of an existing assignment.

Parameters

name (str) – the name of the new solution cell
notebook (str) – the name of an existing notebook
assignment (str) – the name of an existing assignment
**kwargs – additional keyword arguments for SolutionCell

Returns

solution_cell

Return type

SolutionCell

find_solution_cell(name, notebook, assignment)[source]¶

Find a solution cell in a particular notebook of an assignment.

Parameters

name (string) – the name of the solution cell
notebook (string) – the name of the notebook
assignment (string) – the name of the assignment

Returns

solution_cell

Return type

SolutionCell

update_or_create_solution_cell(name, notebook, assignment, **kwargs)[source]¶

Update an existing solution cell in a notebook of an assignment, or create the solution cell if it does not exist.

Parameters

name (str) – the name of the solution cell
notebook (str) – the name of the notebook
assignment (str) – the name of the assignment
**kwargs – additional keyword arguments for SolutionCell

Return type

solution_cell

add_source_cell(name, notebook, assignment, **kwargs)[source]¶

Add a new source cell to an existing notebook of an existing assignment.

Parameters

name (string) – the name of the new source cell
notebook (string) – the name of an existing notebook
assignment (string) – the name of an existing assignment
**kwargs – additional keyword arguments for SourceCell

Returns

source_cell

Return type

SourceCell

find_source_cell(name, notebook, assignment)[source]¶

Find a source cell in a particular notebook of an assignment.

Parameters

name (str) – the name of the source cell
notebook (str) – the name of the notebook
assignment (str) – the name of the assignment

Returns

source_cell

Return type

SourceCell

update_or_create_source_cell(name, notebook, assignment, **kwargs)[source]¶

Update an existing source cell in a notebook of an assignment, or create the source cell if it does not exist.

Parameters

name (str) – the name of the source cell
notebook (str) – the name of the notebook
assignment (str) – the name of the assignment
**kwargs – additional keyword arguments for SourceCell

Return type

source_cell

add_task_cell(name, notebook, assignment, **kwargs)[source]¶

Add a new task cell to an existing notebook of an existing assignment.

Parameters

name (str) – the name of the new solution cell
notebook (str) – the name of an existing notebook
assignment (str) – the name of an existing assignment
**kwargs – additional keyword arguments for TaskCell

Return type

solution_cell

find_task_cell(name, notebook, assignment)[source]¶

Find a task cell in a particular notebook of an assignment.

Parameters

name (string) – the name of the solution cell
notebook (string) – the name of the notebook
assignment (string) – the name of the assignment

Returns

solution_cell

Return type

TaskCell

update_or_create_task_cell(name, notebook, assignment, **kwargs)[source]¶

Update an existing task cell in a notebook of an assignment, or create the solution cell if it does not exist.

Parameters

name (string) – the name of the solution cell
notebook (string) – the name of the notebook
assignment (string) – the name of the assignment
**kwargs – additional keyword arguments for TaskCell

Returns

task_cell

Return type

TaskCell

find_graded_cell(name, notebook, assignment)[source]¶

Find a graded cell in a particular notebook of an assignment. This can be either a GradeCell or a TaskCell

Parameters

name (str) – the name of the grade cell
notebook (str) – the name of the notebook
assignment (str) – the name of the assignment

Return type

grade_cell

add_submission(assignment, student, **kwargs)[source]¶

Add a new submission of an assignment by a student.

This method not only creates the high-level submission object, but also mirrors the entire structure of the existing assignment. Thus, once this method has been called, the new submission exists and is completely ready to be filled in with grades and comments.

Parameters

assignment (str) – the name of an existing assignment
student (str) – the name of an existing student
**kwargs – additional keyword arguments for SubmittedAssignment

Return type

submission

find_submission(assignment, student)[source]¶

Find a student’s submission for a given assignment.

Parameters

assignment (string) – the name of an assignment
student (string) – the unique id of a student

Returns

submission

Return type

SubmittedAssignment

update_or_create_submission(assignment, student, **kwargs)[source]¶

Update an existing submission of an assignment by a given student, or create a new submission if it doesn’t exist.

See add_submission() for additional details.

Parameters

assignment (str) – the name of an existing assignment
student (str) – the name of an existing student
**kwargs – additional keyword arguments for SubmittedAssignment

Return type

submission

grant_extension(assignment, student, minutes=0, hours=0, days=0, weeks=0)[source]¶

Gives an extension to a student for an assignment.

Note that extensions do not stack: if you call this method multiple times for the same assignment and student, the extension will be replaced. So if you want to extend an assignment by two days, and then another day, you will need to call this method with days=3 the second time.

If you do not provide any of the time arguments (minutes, hours, days, weeks), then any existing extension will be removed.

Parameters

assignment (string) – the name of an assignment
student (string) – the unique id of a student
minutes (number (default=0)) – The number of minutes in the extension
hours (number (default=0)) – The number of hours in the extension
days (number (default=0)) – The number of days in the extension
weeks (number (default=0)) – The number of weeks in the extension

remove_submission(assignment, student)[source]¶

Removes a submission from the database.

Parameters

assignment (string) – the name of an assignment
student (string) – the name of a student

remove_submission_notebook(notebook, assignment, student)[source]¶

Removes a submitted notebook from the database.

Parameters

notebook (string) – the name of a notebook
assignment (string) – the name of an assignment
student (string) – the name of a student

assignment_submissions(assignment)[source]¶

Find all submissions of a given assignment.

Parameters: assignment (string) – the name of an assignment
Returns: submissions – A list of SubmittedAssignment objects
Return type: list

notebook_submissions(notebook, assignment)[source]¶

Find all submissions of a given notebook in a given assignment.

Parameters

notebook (string) – the name of a notebook
assignment (string) – the name of an assignment

Returns

submissions – A list of SubmittedNotebook objects

Return type

list

student_submissions(student)[source]¶

Find all submissions by a given student.

Parameters: student (string) – the student’s unique id
Returns: submissions – A list of SubmittedAssignment objects
Return type: list

find_submission_notebook(notebook, assignment, student)[source]¶

Find a particular notebook in a student’s submission for a given assignment.

Parameters

notebook (str) – the name of a notebook
assignment (str) – the name of an assignment
student (str) – the unique id of a student

Return type

notebook

find_submission_notebook_by_id(notebook_id)[source]¶

Find a submitted notebook by its unique id.

Parameters: notebook_id (string) – the unique id of the submitted notebook
Returns: notebook
Return type: SubmittedNotebook

find_grade(grade_cell, notebook, assignment, student)[source]¶

Find a particular grade in a notebook in a student’s submission for a given assignment.

Parameters

grade_cell (str) – the name of a grade or task cell
notebook (str) – the name of a notebook
assignment (str) – the name of an assignment
student (str) – the unique id of a student

Return type

grade

find_grade_by_id(grade_id)[source]¶

Find a grade by its unique id.

Parameters: grade_id (string) – the unique id of the grade
Returns: grade
Return type: Grade

find_comment(solution_cell, notebook, assignment, student)[source]¶

Find a particular comment in a notebook in a student’s submission for a given assignment.

Parameters

solution_cell (str) – the name of a solution or task cell
notebook (str) – the name of a notebook
assignment (str) – the name of an assignment
student (str) – the unique id of a student

Return type

comment

find_comment_by_id(comment_id)[source]¶

Find a comment by its unique id.

Parameters: comment_id (string) – the unique id of the comment
Returns: comment
Return type: Comment

average_assignment_score(assignment_id)[source]¶

Compute the average score for an assignment.

Parameters: assignment_id (string) – the name of the assignment
Returns: score – The average score
Return type: float

average_assignment_code_score(assignment_id)[source]¶

Compute the average code score for an assignment.

Parameters: assignment_id (string) – the name of the assignment
Returns: score – The average code score
Return type: float

average_assignment_written_score(assignment_id)[source]¶

Compute the average written score for an assignment.

Parameters: assignment_id (string) – the name of the assignment
Returns: score – The average written score
Return type: float

average_assignment_task_score(assignment_id)[source]¶

Compute the average task score for an assignment.

Parameters: assignment_id (string) – the name of the assignment
Returns: score – The average task score
Return type: float

average_notebook_score(notebook_id, assignment_id)[source]¶

Compute the average score for a particular notebook in an assignment.

Parameters

notebook_id (str) – the name of the notebook
assignment_id (str) – the name of the assignment

Returns

The average notebook score

Return type

score

average_notebook_code_score(notebook_id, assignment_id)[source]¶

Compute the average code score for a particular notebook in an assignment.

Parameters

notebook_id (str) – the name of the notebook
assignment_id (str) – the name of the assignment

Returns

The average notebook code score

Return type

score

average_notebook_written_score(notebook_id, assignment_id)[source]¶

Compute the average written score for a particular notebook in an assignment.

Parameters

notebook_id (str) – the name of the notebook
assignment_id (str) – the name of the assignment

Returns

The average notebook written score

Return type

score

average_notebook_task_score(notebook_id, assignment_id)[source]¶

Compute the average task score for a particular notebook in an assignment.

Parameters

notebook_id (str) – the name of the notebook
assignment_id (str) – the name of the assignment

Returns

The average notebook task score

Return type

score

student_dicts()[source]¶

Returns a list of dictionaries containing student data. Equivalent to calling to_dict() for each student, except that this method is implemented using proper SQL joins and is much faster.

Returns: students – A list of dictionaries, one per student
Return type: list

submission_dicts(assignment_id)[source]¶

Returns a list of dictionaries containing submission data. Equivalent to calling to_dict() for each submission, except that this method is implemented using proper SQL joins and is much faster.

Parameters: assignment_id (string) – the name of the assignment
Returns: submissions – A list of dictionaries, one per submitted assignment
Return type: list

notebook_submission_dicts(notebook_id, assignment_id)[source]¶

Returns a list of dictionaries containing submission data. Equivalent to calling to_dict() for each submission, except that this method is implemented using proper SQL joins and is much faster.

Parameters

notebook_id (string) – the name of the notebook
assignment_id (string) – the name of the assignment

Returns

submissions – A list of dictionaries, one per submitted notebook

Return type

list

Installation¶

The nbgrader system and command line tools¶

You may install the current version of nbgrader which includes the grading system and command line tools:

pip install nbgrader

Or, if you use Anaconda:

conda install jupyter
conda install -c conda-forge nbgrader

nbgrader extensions¶

Take note: If you install nbgrader via Anaconda the nbgrader extensions will be installed and enabled for you upon installation. See the Installation options and Disabling extensions sections below for more information on changing the default installation option --sys-prefix or disabling one or more extensions.

Installing and activating extensions¶

You can install the nbgrader extensions for Jupyter notebook. Previously this was done using the nbgrader extension install command. However, moving forward this is done using the jupyter nbextension and jupyter serverextension commands.

To install and enable all nbextensions (assignment list, create assignment, formgrader, and validate) run:

jupyter nbextension install --sys-prefix --py nbgrader --overwrite
jupyter nbextension enable --sys-prefix --py nbgrader
jupyter serverextension enable --sys-prefix --py nbgrader

To work properly, the assignment list, formgrader, and validate extensions require both the nbextension and serverextension. The create assignment extension only has an nbextension part.

Installation options¶

When installed/enabled with the --sys-prefix option, the nbextensions and serverextension will be installed and enabled for anyone using the particular Python installation or conda environment where nbgrader is installed. If that Python installation is available system-wide, all users will immediately be able to use the nbgrader extensions.

There are a number of ways you may need to customize the installation:

To install or enable the nbextensions/serverextension for just the current user, run the above commands with --user instead of --sys-prefix:

jupyter nbextension install --user --py nbgrader --overwrite
jupyter nbextension enable --user --py nbgrader
jupyter serverextension enable --user --py nbgrader

To install or enable the nbextensions/serverextension for all Python installations on the system, run the above commands with --system instead of --sys-prefix:

jupyter nbextension install --system --py nbgrader --overwrite
jupyter nbextension enable --system --py nbgrader
jupyter serverextension enable --system --py nbgrader

You can also use the --overwrite option along with the jupyter nbextension install command to overwrite existing nbgrader extension installation files, typically used when updating nbgrader, for example:
```
jupyter nbextension install --sys-prefix --overwrite --py nbgrader
```

Previous versions of nbgrader required each user on a system to enable the nbextensions; this is no longer needed if the --sys-prefix option is used for a system-wide python or the --system option is used.

Disabling extensions¶

You may want to only install one of the nbgrader extensions. To do this, follow the above steps to install everything and then disable the extension you don’t need. For example, to disable the Assignment List extension:

jupyter nbextension disable --sys-prefix assignment_list/main --section=tree
jupyter serverextension disable --sys-prefix nbgrader.server_extensions.assignment_list

or to disable the Create Assignment extension:

jupyter nbextension disable --sys-prefix create_assignment/main

or to disable the Formgrader extension:

jupyter nbextension disable --sys-prefix formgrader/main --section=tree
jupyter serverextension disable --sys-prefix nbgrader.server_extensions.formgrader

or to disable the Course List extension:

jupyter nbextension disable --sys-prefix course_list/main --section=tree
jupyter serverextension disable --sys-prefix nbgrader.server_extensions.course_list

For example lets assume you have installed nbgrader via Anaconda (meaning all extensions are installed and enabled with the --sys-prefix flag, i.e. anyone using the particular Python installation or conda environment where nbgrader is installed). But you only want the create assignment extension available to a specific user and not everyone else. First you will need to disable the create assignment extension for everyone else:

jupyter nbextension disable --sys-prefix create_assignment/main

Log in with the specific user and then enable the create assignment extension only for that user:

jupyter nbextension enable --user create_assignment/main

Finally to see all installed nbextensions/server extensions, run:

jupyter nbextension list
jupyter serverextension list

For further documentation on these commands run:

jupyter nbextension --help-all
jupyter serverextension --help-all

For advanced instructions on installing the assignment list extension please see the advanced installation instructions.

Quick start¶

To get up and running with nbgrader quickly, you can create an example directory with example course files in it by running the nbgrader quickstart command:

nbgrader quickstart course_id

Where you should replace course_id with the name of your course. For further details on how the quickstart command works, please run:

nbgrader quickstart --help

For an explanation of how this directory is arranged, and what the different files are in it, continue reading on in The philosophy and the approach.

What is nbgrader?¶

This page describes the internals of nbgrader from a rather technical standpoint. It is needed not just for developers, but also for administrators which need to build advanced deployments, possibly customizing various components in the process.

A companion (and minor prerequisite) to this is the Jupyter conceptual introduction <https://jupyterhub.readthedocs.io/en/latest/getting-started/what-is-jupyterhub.html> (once it is released).

Working with nbgrader¶

When nbgrader was first written, all interactions were done on the command line. Later on it acquired notebook extensions to enable interaction from within a jupyter notebook. These notes will refer to the command-line interface, and we’ll discuss the UI interactions later. Over time, this and other improvements have been added, but because of the simple roots, it is not too difficult to dig deeper and innovate to your specific case.

Nbgrader format¶

Nbgrader uses the standard Jupyter notebook format, .ipynb. This is the primary power of nbgrader: since it uses a standard format that’s used for other types of computation. Since it is not special to nbgrader, it is immediately usable on other systems (running on your own computer, uploading to Google Colab, using anything from the Jupyter ecosystem). Thus, to begin developing nbgrader notebooks, nothing special is needed. They are directly portable to research code, other projects, etc. The skills learned in doing nbgrader assignments is portable to other real-world projects.

All of the files are stored directly on a filesystem, so that they are easy to manipulate and understand. The text (well, JSON) based notebook format is well supported by Jupyter tools for manipulation, making it easy to script other tools with them.

Jupyter notebooks can have metadata - both notebook-wide and per-cell. This is used to track the progress of the assignment. For example, this tracks cells that are read-only. However, since the metadata is within the notebook file itself - which is given to the student - one can’t prevent the student from editing the notebook file to change anything within it, since they have direct access to the file within their server. More on the effects of this later.

The standard Jupyter stack and notebook format is also one of the downsides, compared to other learning management systems (depending on if you consider this a disadvantage…). Because this is not a “captive system” that segments users from the data they are working on, you can’t easily limit access to the underlying operating system or notebook file. For example, it is very difficult to prevent access to the actual .ipynb source file, if you wanted to do that. The student actually has access to the computational environment (subject to the limits of the JupyterHub/spawner you set up, which can limit students to their own environment). However, if you design your course considering this, we hope you’ll see that this is an advantage.

Course directory¶

From the instructor side, all data is stored in files in the course directory. Within this directory, files have the general structure {nbgrader_step}/{student_id}/{assignment_id}/{notebook}.ipynb. The {nbgrader_step} indicates the progress of the notebook:

source/ contains the raw source notebooks made by the instructor. You run nbgrader generate_assignment to produce the…
release/ directory. This contains a version with the “hidden tests” removed and various cells marked read-only. For both the source and release steps, the student_id part of the path is missing, since the notebooks are not personalized per student.
The release version gets copied to the students, and copied back. This can be through the “exchange” (below), or some other technique.
The submitted/ step contains the students submitted versions. By running nbgrader autograde, the files are autograded (see below) and copied to the…
autograded directory. Autograding is described below, but basically consists of executing the notebook and looking for failed cells. autograded stores the executed notebook. One can also use this for manual grading. Running generate_feedback, one gets…
feedback. This stores a .html (not .ipynb) file with the executed notebook, points, and comments from manual grading. This can be returned to the students.

But, there is a database in gradebook.db which stores students and grades. Less obviously, it stores the assignments and the contents of hidden or immutable cells, such as the contents of the hidden tests or read-only cells. This is used to restore these cells when students return them.

Most operations primarily look at the filesystem to see the current state, so that it is easy to understand and manipulate the internals. In other words, just by manipulating the files on disk, you can control all of the nbgrader steps. The gradebook is a sort of secondary source of information most of the time.

nbgrader generate_assignment¶

generate_assignment converts the release notebook file from the source. Like all of these commands, it is implemented as converters that implement various filters of the notebook source, in this case reading from a path in source/ and writing to a path in release/. Exact operations can be found within the nbgrader.converters path.

The main effect of generate_assignment is to strip all output cells and remove ### BEGIN SOLUTION AND ### BEGIN HIDDEN TEST blocks, and record the contents of various read-only cells in the database.

nbgrader autograde¶

This is described below, but the core tasks are to re-insert the read-only cells (replace them with the known-good versions), and execute the entire notebook. Note that there is a separate manual grading step, but this isn’t a nbgrader command, but done in the formgrader interface. This is described more below.

nbgrader generate_feedback¶

This will take any feedback created during manual grading, for all student submissions in this assignments, and create a .html file. This uses the same machinery that nbconvert uses, but with a custom template that adds in information about grading, points, etc.

Collecting and releasing: collectors and the exchange¶

There are various ways to release assignments to students and get them back: something done out-of-nbgrader by you, using the filesystem exchange, or (in the future) other exchanges.

Out-of-band exchange¶

Originally, nbgrader had no built-in way of distributing assignments. You would upload the assignments to your other course management system, students would download, do it on their own computers (or wherever - it’s just a standard Jupyter notebook file, of course), and upload to the course management system again. The system hopefully allows you to download the files of all students in bulk, organized in a particular fashion (sorted by student username, for example). You can then use a nbgrader collector to import the tree of student notebooks back into the submitted step of the course directory.

This is simple, effective, and relatively foolproof and secure - but requires the student to create their own Jupyter environment to do the assignment. These days, Jupyter has the promise of the cloud, so there is some way to manage this for the students:

Filesystem exchange¶

Let’s say that you have a JupyterHub web server with accounts for all students. All students have access to the same system via JupyterHub, properly segmented into user accounts. The filesystem exchange allows you to distribute the assignments, students to submit completed assignments, and release feedback. The filesystem exchange is simple but effective.

It is structured as:

outbound/, containing assignments released to students.
- Organized as outbound/{assignment_id}/{notebook}.ipynb. Other data files can be distributed along with the notebook.
- Files are copied from {course_dir}/release/ to {exchange_dir}/outbound/ by the nbgrader release_assignment command.
inbound/, containing assignments the student is submitting. It should be writeable, but not listable, by students (-wx UNIX permissions).
- Organized as inbound/{studend_id}+{assignment_id}+{timestamp}+{random_string}/{notebook}.ipynb. One of these directories contains one submission of the assignment. The protection of them is within the random string, which is described below.
- Files are copied from {exchange_dir}/inbound/ to {course_dir}/submitted/ by the nbgrader fetch_assignment command.
feedback/, containing feedback to the students. This should be traversable by students, but not listable (--x). The files inside should be readable (r--).
- Organized as feedback/{hash}.html, where {hash} is a hash of notebook contents and timestamp of submission. It serves as a key which is known to the student but not to other students, so that they can identify their notebook and retrieve it.
- Files are copied from {course_dir}/feedback/ to {exchange_dir}/feedback/ by the nbgrader release_feedback command.

The filesystem exchange relies on certain UNIX filesystem semantics: if a user has write and execute permissions on a directory, they can create files inside of it but not list other files in there. If each file has an unpredictable name (e.g. by a random string), students can not access each others files (this is used for submitting assignments). Furthermore, they can access files they do know the names of (this is used for retrieving feedback). In order for these assumptions to apply, students must access the filesystem under different numeric UNIX user ids (UIDs).

The filesystem exchange isn’t limited to just one computer, though. Network filesystems exist and have the necessary UNIX semantics - in particular, the Network Filesystem (NFS). This can easily be used to mount an exchange directory on multiple computers, so that students can be distributed among multiple computers within a cluster. However, this requires a consistent mapping to UIDs across the cluster. This is not difficult to do, but if often not the way that “cloud stuff” works by default.

The default filesystem exchange path is /srv/nbgrader/exchange. In a UNIX file system, this is by default owned by the root user, so you will need to use a bit of knowledge to set things up properly.

Other exchanges¶

While a network-mounted filesystem exchange can work, it still is limited to UNIX filesystem semantics, which is quite limited. There are API-based network exchanges under development, which will allow a true decoupling of the student environment from the course management.

More generally, as part of that work, a pluggable exchange concept is being developed, so that the exchange is a class which can be replaced by custom implementations.

Student directories¶

The basic principle is that the student copy of assignments are copied to and from the student’s home directory (on more precisely, the working directory of the notebook server). Once they are in the student directory, they are accessed just like any other notebooks or data the student can access.

Autograding¶

Autograding is very simple in principle: run the notebook. The actual effect is no different than the “Restart and run all cells” functionality within the Jupyter interface.

The difference is that, after running, it looks for cells that have an error output. If any of these cells are marked as “autograder tests”, then these cells have a point value, and that point value is subtracted. Error output is simple any text on the standard error stream, which is saved separately within the notebook output from the standard output stream. It is up to the Jupyter kernel to write an error message to the standard error stream, otherwise autograder doesn’t work (this has been a problem with a few languages kernels in the past).

TODO: partial credit. If a autograder test cell outputs a single number to the standard output stream, then it will use that as the number of points. However, this could always be simulated by dividing the autograded task into multiple cells.

Validation¶

Validation is very related to autograding. There is a button on the student interface marked “validate”, which executes the student version of the notebook from top to bottom, and reports any errors. This is exactly equivalent to “Restart and run all”, but doesn’t stop on errors. Since all it can access is the actual notebook file the student has, it can not take into account the hidden tests. If an instructor wants a test to be visible to the students.

There is currently no support for inserting hidden tests into the notebook file (perhaps you could in a hidden cell, but since the student actually has the file… it’s not going to be hidden to anyone willing to do a bit of exploration).

Manual grading¶

After autograding, there is a web UI (via the formgrader extension) to do manual grading. This allows one to see the output from autograding, give comments, adjust points, etc. There are also purely manually graded exercises.

The output from manual grading is only stored in gradebook.db, and is merged into the final output at the feedback step.

gradebook.db and student management¶

The gradebook or database is stored (by default) at gradebook.db at the root of the course directory. Out of the box, it is sqlite3, but can be other database systems, too.

First, the gradebook stores student mappings. It stores a student_id (string) that is the name used on the filesystem for each student. It can also store a firstname/lastname/email for each student, but it doesn’t try to replace a complex student management system.

The database also stores assignments and their cells. For example, it stores the contents of read-only cells, and autograder tests cells, which get re-inserted into the notebook before the autograde step. Cells are stored by the cell ID, which is in the cell metadata (cell metadata is a ipynb-format native concept). The autograder step looks at the database and re-inserts data based on the cell ID.

In the formgrader “manual grading” interface, the instructor can manually grade assignments (after autograding), and these points + comments are added to the database.

Grades can be exported in csv format. You can also build other exporters, which access the database and export somehow - to a file, or perhaps other fancy things like uploading directly.

Feedback¶

Feedback is a HTML file, basically like a rendering of the notebook using nbconvert. However, it adds in points and feedback.

Historically, feedback was just generated, and it was up to the teacher to distribute it somehow (for example, uploading to the course management system or scripting copying it into users home directories). Now, using the exchange, there can be automatically distributed. This is described above.

Web extensions¶

Most of the above originally was handled via a command line interface. But now there are several interfaces directly from Jupyter, and these are essentially the “default” ways of using nbgrader.

The Assignment list extension serves as the student-facing interface for the notebook file browser view. It fetches assignments from the exchange directory, allows students to open them, and submit them back to the exchange. This is for the Jupyter notebook file-browser view

The formgrader extension is the instructor-facing interface accessible from the file browser view. It allows the instructor to browse assignments, open them, manage students, etc. This is for the Jupyter notebook file-browser view.

The validate extension is a student-facing for the notebook view that does validation. Basically, it is the same as “Restart and run all cells” but it shows errors a little bit nicer.

The create assignment extension is an instructor-facing for the notebook view. It provides a toolbar that allows you to edit cell metadata.

Currently, these only work for the Jupyter notebook interface, not JupyterLab. This is a point under development.

Customizing how the student version of an assignment looks¶

See also

Creating and grading assignments: Documentation for nbgrader generate_assignment, nbgrader autograde, and nbgrader generate_feedback.
nbgrader generate assignment: Command line options for nbgrader generate_assignment
The nbgrader_config.py file: Details on nbgrader_config.py

“Autograded answer” cells¶

Default behavior¶

By default, nbgrader generate_assignment will replace regions beginning with BEGIN SOLUTION and END SOLUTION comment delimeters with:

# YOUR CODE HERE
raise NotImplementedError

Note that the code stubs will be properly indented based on the indentation of the solution delimeters. For example, if your original code is:

def foo(bar):
    """Prints `bar`."""
    ### BEGIN SOLUTION
    print(bar)
    ### END SOLUTION

then the solution region will become:

def foo(bar):
    """Prints `bar`."""
    # YOUR CODE HERE
    raise NotImplementedError

These solution comment delimeters are independent of the programming language used and the number of comment characters used in the source notebook. For example, this default will work for both Python:

def foo(bar):
    """Prints `bar`."""
    ### BEGIN SOLUTION
    print(bar)
    ### END SOLUTION

and JavaScript:

function foo (bar){
    // BEGIN SOLUTION
    console.log(bar);
    // END SOLUTION
}

If the solution delimeters aren’t present, nbgrader will replace the entire contents of all manually graded cells and autograded cells with the above code stub (if it is a code cell) or a text stub (if it is a Markdown cell), the default of which is YOUR ANSWER HERE.

Changing the defaults¶

If you need to change these defaults (e.g., if your class doesn’t use Python, or isn’t taught in English), the values can be configured in the nbgrader_config.py file. Most relevant is the code_stub option to the ClearSolutions preprocessor, which is the part of nbgrader that actually clears the solutions when producing the student version of the notebook.

The solution delimeters are independent of the programming language used, however the code stub depends on the language of the notebook, the default of which is Python. You can specify solution delimeters for any languages you want by setting the ClearSolutions.begin_solution_delimeter, ClearSolutions.end_solution_delimeter, and ClearSolutions.code_stub config options, thus allowing you to include notebooks of different languages within the same assignment:

c = get_config()
c.ClearSolutions.begin_solution_delimeter = "BEGIN MY SOLUTION"
c.ClearSolutions.end_solution_delimeter = "END MY SOLUTION"
c.ClearSolutions.code_stub = {
    "python": "# your code here\nraise NotImplementedError",
    "javascript": "// your code here\nthrow new Error();"
}

Note

Note that the code stub itself doesn’t have to cause an error (though that is the easiest thing to do, in my opinion) – it all depends on how you write your test cases. The only constraint is that when autograding happens, the behavior is such that:

If the tests pass, the student gets full credit.
If the tests fail, the student gets no credit.

So if the student hasn’t given an answer, the tests should ideally fail by default. How they fail is totally up to how you write your test cases.

Similarly, the text stub that the contents of Markdown cells get replaced with can be configured through the ClearSolutions.text_stub option:

c.ClearSolutions.text_stub = "Please replace this text with your response."

“Autograder tests” cells with hidden tests¶

New in version 0.5.0.

Default behavior¶

By default, nbgrader generate_assignment will remove tests wrapped within the BEGIN HIDDEN TESTS and END HIDDEN TESTS comment delimeters, for example:

assert squares(1) = [1]
### BEGIN HIDDEN TESTS
assert squares(2) = [1, 4]
### END HIDDEN TESTS

will be released as:

assert squares(1) = [1]

These comment delimeters are independent of the programming language used and the number of comment characters used in the source notebook. For example, this default will work for both Python:

assert squares(1) = [1]
### BEGIN HIDDEN TESTS
assert squares(2) = [1, 4]
### END HIDDEN TESTS

and JavaScript:

function assert(answer, expected, msg) {
    correct = ...;  // validate the answer
    if (!correct) {
        throw msg || "Incorrect answer";
    }
}

assert(squares(1), [1]);
// BEGIN HIDDEN TESTS
assert(squares(2), [1, 4]);
// END HIDDEN TESTS

Note

Keep in mind that wrapping all tests (for an “Autograder tests” cell) in this special syntax will remove all these tests in the release version and the students will only see a blank cell. It is recommended to have at least one or more visible tests, or a comment in the cell for the students to see.

Changing the defaults¶

If you need to change these defaults (e.g., if your class isn’t taught in English), the values can be configured in the nbgrader_config.py file. Most relevant are the options to the ClearHiddenTests preprocessor, which is the part of nbgrader that actually removes the tests when producing the student version of the notebook.

You can specify hidden test delimeters you want by setting the ClearHiddenTests.begin_test_delimeter and ClearHiddenTests.end_test_delimeter config options:

c = get_config()
c.ClearHiddenTests.begin_test_delimeter = "VERBORGE TOESTE BEGIN"
c.ClearHiddenTests.end_test_delimeter = "VERBORGE TOESTE EINDIG"

The `nbgrader_config.py` file¶

See also

Configuration options: A list of all config options for nbgrader_config.py.

The nbgrader_config.py file is the main place for you to specify options for configuring nbgrader. Normally, it should be located in the same directory as where you are running the nbgrader commands, or you can place it in one of a number of other locations on your system. These locations correspond to the configuration directories that Jupyter itself looks in; you can find out what these are by running jupyter --paths.

Things get a bit more complicated in certain setups, so this document aims to clarify how to setup the nbgrader_config.py file in multiple different scenarios.

Using `nbgrader_config.py`¶

To set a configuration option in the config file, you need to use the c variable which actually stores the config. For example:

c = get_config()
c.CourseDirectory.course_id = "course101"

To get an example config file, you can run nbgrader generate_config.

Use Case 1: nbgrader and `jupyter notebook` run in the same directory¶

The easiest way to use nbgrader and the formgrader extension is to run both from the same directory. For example:

nbgrader quickstart ./course101
cd ./course101
jupyter notebook

In this case, there should be a nbgrader_config.py file in the directory ./course101, which corresponds both to the directory where the notebook is running and the directory where the nbgrader commands will be run.

As mentioned above, you can actually put the nbgrader_config.py file in any of the directories listed by jupyter --paths in the “Config” section.

Use Case 2: nbgrader and `jupyter notebook` run in separate directories¶

Warning

The nbgrader course directory must be a subdirectory of where you run the Jupyter notebook.

A common use case is to run the notebook server from the root of your home directory, which is likely not the place where you will be running nbgrader from. In this case, you will need to tell the nbgrader extensions—which run as part of the notebook server—where to find your course directory. In this case, you want two nbgrader_config.py files: one for your main course directory (where you run the nbgrader commands) and one that specifies for the notebook where the course directory is.

For example, the nbgrader_config.py that the notebook knows about could be placed in ~/.jupyter/nbgrader_config.py, and it would include a path to where the main course directory is:

c = get_config()
c.CourseDirectory.root = "/path/to/course/directory"

Then you would additionally have a config file at /path/to/course/directory/nbgrader_config.py.

Use Case 3: nbgrader and JupyterHub¶

See also

Using nbgrader with JupyterHub: Further information on using nbgrader with JupyterHub.

The setup of nbgrader_config.py files gets a bit more complicated when you are running a shared server with JupyterHub. In this case, you will likely need (at least) three separate nbgrader_config.py files:

A global nbgrader_config.py (for example, placed in a global path like /usr/local/etc/jupyter or /etc/jupyter, but check jupyter --paths to see which ones are valid on your system). This global file should include information relevant to all students, instructors, and formgraders, such as the location of the exchange directory.
An nbgrader_config.py that tells the notebook server running the formgrader where the course directory is located (as in Use Case 2). The options in this config file will only be relevant for the formgrader, and not any other user accounts.
An nbgrader_config.py file in the course directory itself. The options in this config file will only be relevant for the formgrader, and not any other user accounts.

Configuration options¶

See also

The nbgrader_config.py file: Details on how to setup the nbgrader_config.py file.

These options can be set in nbgrader_config.py, or at the command line when you start it.

Application.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

Application.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

Application.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

Application.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

Application.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

JupyterApp.answer_yesBool

Default: False

Answer yes to any prompts.

JupyterApp.config_fileUnicode

Default: ''

Full path of a config file.

JupyterApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

JupyterApp.generate_configBool

Default: False

Generate default config file.

JupyterApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

JupyterApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

JupyterApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

JupyterApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

JupyterApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

NbGrader.answer_yesBool

Default: False

Answer yes to any prompts.

NbGrader.config_fileUnicode

Default: ''

Full path of a config file.

NbGrader.config_file_nameUnicode

Default: ''

Specify a config file to load.

NbGrader.generate_configBool

Default: False

Generate default config file.

NbGrader.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

NbGrader.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

NbGrader.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

NbGrader.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

NbGrader.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

NbGrader.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

NbGraderApp.answer_yesBool

Default: False

Answer yes to any prompts.

NbGraderApp.config_fileUnicode

Default: ''

Full path of a config file.

NbGraderApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

NbGraderApp.generate_configBool

Default: False

Generate default config file.

NbGraderApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

NbGraderApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

NbGraderApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

NbGraderApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

NbGraderApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

NbGraderApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ExchangeFactory.collectType

Default: 'nbgrader.exchange.default.collect.ExchangeCollect'

A plugin for collecting assignments.

ExchangeFactory.exchangeType

Default: 'nbgrader.exchange.default.exchange.Exchange'

A plugin for exchange.

ExchangeFactory.fetch_assignmentType

Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...

A plugin for fetching assignments.

ExchangeFactory.fetch_feedbackType

Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'

A plugin for fetching feedback.

ExchangeFactory.listType

Default: 'nbgrader.exchange.default.list.ExchangeList'

A plugin for listing exchange files.

ExchangeFactory.release_assignmentType

Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...

A plugin for releasing assignments.

ExchangeFactory.release_feedbackType

Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...

A plugin for releasing feedback.

ExchangeFactory.submitType

Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

A plugin for submitting assignments.

CourseDirectory.assignment_idUnicode

Default: ''

The assignment name. This MUST be specified, either by setting the config option, passing an argument on the command line, or using the –assignment option on the command line.

CourseDirectory.autograded_directoryUnicode

Default: 'autograded'

The name of the directory that contains assignment submissions after they have been autograded. This corresponds to the nbgrader_step variable in the directory_structure config option.

CourseDirectory.course_idUnicode

Default: ''

A key that is unique per instructor and course. This can be specified, either by setting the config option, or using the –course option on the command line.

CourseDirectory.db_urlUnicode

Default: ''

URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root> is another configurable variable.

CourseDirectory.directory_structureUnicode

Default: '{nbgrader_step}/{student_id}/{assignment_id}'

Format string for the directory structure that nbgrader works over during the grading process. This MUST contain named keys for ‘nbgrader_step’, ‘student_id’, and ‘assignment_id’. It SHOULD NOT contain a key for ‘notebook_id’, as this will be automatically joined with the rest of the path.

CourseDirectory.feedback_directoryUnicode

Default: 'feedback'

The name of the directory that contains assignment feedback after grading has been completed. This corresponds to the nbgrader_step variable in the directory_structure config option.

CourseDirectory.groupsharedBool

Default: False

Make all instructor files group writeable (g+ws, default g+r only) and exchange directories group readable/writeable (g+rws, default g=nothing only ) by default. This should only be used if you carefully set the primary groups of your notebook servers and fully understand the unix permission model. This changes the default permissions from 444 (unwriteable) to 664 (writeable), so that other instructors are able to delete/overwrite files.

CourseDirectory.ignoreList

Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']

List of file names or file globs. Upon copying directories recursively, matching files and directories will be ignored with a debug message.

CourseDirectory.includeList

Default: ['*']

List of file names or file globs. Upon copying directories recursively, non matching files will be ignored with a debug message.

CourseDirectory.max_file_sizeInt

Default: 100000

Maximum size of files (in kilobytes; default: 100Mb). Upon copying directories recursively, larger files will be ignored with a warning.

CourseDirectory.notebook_idUnicode

Default: '*'

File glob to match notebook names, excluding the ‘.ipynb’ extension. This can be changed to filter by notebook.

CourseDirectory.release_directoryUnicode

Default: 'release'

The name of the directory that contains the version of the assignment that will be released to students. This corresponds to the nbgrader_step variable in the directory_structure config option.

CourseDirectory.rootUnicode

Default: ''

The root directory for the course files (that includes the source, release, submitted, autograded, etc. directories). Defaults to the current working directory.

CourseDirectory.solution_directoryUnicode

Default: 'solution'

The name of the directory that contains the assignment solution after grading has been completed. This corresponds to the nbgrader_step variable in the directory_structure config option.

CourseDirectory.source_directoryUnicode

Default: 'source'

The name of the directory that contains the master/instructor version of assignments. This corresponds to the nbgrader_step variable in the directory_structure config option.

CourseDirectory.student_idUnicode

Default: '*'

File glob to match student IDs. This can be changed to filter by student. Note: this is always changed to ‘.’ when running nbgrader assign, as the assign step doesn’t have any student ID associated with it. With nbgrader submit, this instead forces the use of an alternative student ID for the submission. See nbgrader submit –help.

If the ID is purely numeric and you are passing it as a flag on the command line, you will need to escape the quotes in order to have it detected as a string, for example –student=””12345””. See:

https://github.com/jupyter/nbgrader/issues/743

for more details.

CourseDirectory.student_id_excludeUnicode

Default: ''

Comma-separated list of student IDs to exclude. Counterpart of student_id.

This is useful when running commands on all students, but certain students cause errors or otherwise must be left out. Works at least for autograde, generate_feedback, and release_feedback.

CourseDirectory.submitted_directoryUnicode

Default: 'submitted'

The name of the directory that contains assignments that have been submitted by students for grading. This corresponds to the nbgrader_step variable in the directory_structure config option.

Authenticator.plugin_classType

Default: 'nbgrader.auth.base.NoAuthPlugin'

A plugin for different authentication methods.

GenerateAssignmentApp.answer_yesBool

Default: False

Answer yes to any prompts.

GenerateAssignmentApp.config_fileUnicode

Default: ''

Full path of a config file.

GenerateAssignmentApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

GenerateAssignmentApp.generate_configBool

Default: False

Generate default config file.

GenerateAssignmentApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

GenerateAssignmentApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

GenerateAssignmentApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

GenerateAssignmentApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

GenerateAssignmentApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

GenerateAssignmentApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

AssignApp.answer_yesBool

Default: False

Answer yes to any prompts.

AssignApp.config_fileUnicode

Default: ''

Full path of a config file.

AssignApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

AssignApp.generate_configBool

Default: False

Generate default config file.

AssignApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

AssignApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

AssignApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

AssignApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

AssignApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

AssignApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

AutogradeApp.answer_yesBool

Default: False

Answer yes to any prompts.

AutogradeApp.config_fileUnicode

Default: ''

Full path of a config file.

AutogradeApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

AutogradeApp.generate_configBool

Default: False

Generate default config file.

AutogradeApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

AutogradeApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

AutogradeApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

AutogradeApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

AutogradeApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

AutogradeApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

FormgradeApp.answer_yesBool

Default: False

Answer yes to any prompts.

FormgradeApp.config_fileUnicode

Default: ''

Full path of a config file.

FormgradeApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

FormgradeApp.generate_configBool

Default: False

Generate default config file.

FormgradeApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

FormgradeApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

FormgradeApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

FormgradeApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

FormgradeApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

FormgradeApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

GenerateFeedbackApp.answer_yesBool

Default: False

Answer yes to any prompts.

GenerateFeedbackApp.config_fileUnicode

Default: ''

Full path of a config file.

GenerateFeedbackApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

GenerateFeedbackApp.generate_configBool

Default: False

Generate default config file.

GenerateFeedbackApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

GenerateFeedbackApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

GenerateFeedbackApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

GenerateFeedbackApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

GenerateFeedbackApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

GenerateFeedbackApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

FeedbackApp.answer_yesBool

Default: False

Answer yes to any prompts.

FeedbackApp.config_fileUnicode

Default: ''

Full path of a config file.

FeedbackApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

FeedbackApp.generate_configBool

Default: False

Generate default config file.

FeedbackApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

FeedbackApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

FeedbackApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

FeedbackApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

FeedbackApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

FeedbackApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ValidateApp.answer_yesBool

Default: False

Answer yes to any prompts.

ValidateApp.config_fileUnicode

Default: ''

Full path of a config file.

ValidateApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ValidateApp.generate_configBool

Default: False

Generate default config file.

ValidateApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ValidateApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ValidateApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ValidateApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ValidateApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ValidateApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ReleaseAssignmentApp.answer_yesBool

Default: False

Answer yes to any prompts.

ReleaseAssignmentApp.config_fileUnicode

Default: ''

Full path of a config file.

ReleaseAssignmentApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ReleaseAssignmentApp.generate_configBool

Default: False

Generate default config file.

ReleaseAssignmentApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ReleaseAssignmentApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ReleaseAssignmentApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ReleaseAssignmentApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ReleaseAssignmentApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ReleaseAssignmentApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ReleaseApp.answer_yesBool

Default: False

Answer yes to any prompts.

ReleaseApp.config_fileUnicode

Default: ''

Full path of a config file.

ReleaseApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ReleaseApp.generate_configBool

Default: False

Generate default config file.

ReleaseApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ReleaseApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ReleaseApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ReleaseApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ReleaseApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ReleaseApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ReleaseFeedbackApp.answer_yesBool

Default: False

Answer yes to any prompts.

ReleaseFeedbackApp.config_fileUnicode

Default: ''

Full path of a config file.

ReleaseFeedbackApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ReleaseFeedbackApp.generate_configBool

Default: False

Generate default config file.

ReleaseFeedbackApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ReleaseFeedbackApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ReleaseFeedbackApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ReleaseFeedbackApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ReleaseFeedbackApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ReleaseFeedbackApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

CollectApp.answer_yesBool

Default: False

Answer yes to any prompts.

CollectApp.config_fileUnicode

Default: ''

Full path of a config file.

CollectApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

CollectApp.generate_configBool

Default: False

Generate default config file.

CollectApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

CollectApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

CollectApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

CollectApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

CollectApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

CollectApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ZipCollectApp.answer_yesBool

Default: False

Answer yes to any prompts.

ZipCollectApp.archive_directoryUnicode

Default: 'archive'

The name of the directory that contains assignment submission files and/or archives (zip) files manually downloaded from a LMS. This corresponds to the collect_step variable in the collect_structure config option.

ZipCollectApp.collect_directory_structureUnicode

Default: '{downloaded}/{assignment_id}/{collect_step}'

Format string for the directory structure that nbgrader works over during the zip collect process. This MUST contain named keys for ‘downloaded’, ‘assignment_id’, and ‘collect_step’.

ZipCollectApp.collector_pluginType

Default: 'nbgrader.plugins.zipcollect.FileNameCollectorPlugin'

The plugin class for processing the submitted file names after they have been extracted into the extracted_directory.

ZipCollectApp.config_fileUnicode

Default: ''

Full path of a config file.

ZipCollectApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ZipCollectApp.downloaded_directoryUnicode

Default: 'downloaded'

The main directory that corresponds to the downloaded variable in the collect_structure config option.

ZipCollectApp.extracted_directoryUnicode

Default: 'extracted'

The name of the directory that contains assignment submission files extracted or copied from the archive_directory. This corresponds to the collect_step variable in the collect_structure config option.

ZipCollectApp.extractor_pluginType

Default: 'nbgrader.plugins.zipcollect.ExtractorPlugin'

The plugin class for extracting the archive files in the archive_directory.

ZipCollectApp.forceBool

Default: False

Force overwrite of existing files.

ZipCollectApp.generate_configBool

Default: False

Generate default config file.

ZipCollectApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ZipCollectApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ZipCollectApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ZipCollectApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ZipCollectApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ZipCollectApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ZipCollectApp.strictBool

Default: False

Skip submitted notebooks with invalid names.

FetchAssignmentApp.answer_yesBool

Default: False

Answer yes to any prompts.

FetchAssignmentApp.config_fileUnicode

Default: ''

Full path of a config file.

FetchAssignmentApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

FetchAssignmentApp.generate_configBool

Default: False

Generate default config file.

FetchAssignmentApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

FetchAssignmentApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

FetchAssignmentApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

FetchAssignmentApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

FetchAssignmentApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

FetchAssignmentApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

FetchApp.answer_yesBool

Default: False

Answer yes to any prompts.

FetchApp.config_fileUnicode

Default: ''

Full path of a config file.

FetchApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

FetchApp.generate_configBool

Default: False

Generate default config file.

FetchApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

FetchApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

FetchApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

FetchApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

FetchApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

FetchApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

FetchFeedbackApp.answer_yesBool

Default: False

Answer yes to any prompts.

FetchFeedbackApp.config_fileUnicode

Default: ''

Full path of a config file.

FetchFeedbackApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

FetchFeedbackApp.generate_configBool

Default: False

Generate default config file.

FetchFeedbackApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

FetchFeedbackApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

FetchFeedbackApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

FetchFeedbackApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

FetchFeedbackApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

FetchFeedbackApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

SubmitApp.answer_yesBool

Default: False

Answer yes to any prompts.

SubmitApp.config_fileUnicode

Default: ''

Full path of a config file.

SubmitApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

SubmitApp.generate_configBool

Default: False

Generate default config file.

SubmitApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

SubmitApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

SubmitApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

SubmitApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

SubmitApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

SubmitApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ListApp.answer_yesBool

Default: False

Answer yes to any prompts.

ListApp.config_fileUnicode

Default: ''

Full path of a config file.

ListApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ListApp.generate_configBool

Default: False

Generate default config file.

ListApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ListApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ListApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ListApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ListApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ListApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ExtensionApp.answer_yesBool

Default: False

Answer yes to any prompts.

ExtensionApp.config_fileUnicode

Default: ''

Full path of a config file.

ExtensionApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ExtensionApp.generate_configBool

Default: False

Generate default config file.

ExtensionApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ExtensionApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ExtensionApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ExtensionApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ExtensionApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ExtensionApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

QuickStartApp.answer_yesBool

Default: False

Answer yes to any prompts.

QuickStartApp.config_fileUnicode

Default: ''

Full path of a config file.

QuickStartApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

QuickStartApp.forceBool

Default: False

Whether to overwrite existing files

QuickStartApp.generate_configBool

Default: False

Generate default config file.

QuickStartApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

QuickStartApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

QuickStartApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

QuickStartApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

QuickStartApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

QuickStartApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ExportApp.answer_yesBool

Default: False

Answer yes to any prompts.

ExportApp.config_fileUnicode

Default: ''

Full path of a config file.

ExportApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

ExportApp.generate_configBool

Default: False

Generate default config file.

ExportApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

ExportApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

ExportApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

ExportApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

ExportApp.plugin_classType

Default: 'nbgrader.plugins.export.CsvExportPlugin'

The plugin class for exporting the grades.

ExportApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

ExportApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

DbBaseApp.answer_yesBool

Default: False

Answer yes to any prompts.

DbBaseApp.config_fileUnicode

Default: ''

Full path of a config file.

DbBaseApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

DbBaseApp.generate_configBool

Default: False

Generate default config file.

DbBaseApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

DbBaseApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

DbBaseApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

DbBaseApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

DbBaseApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

DbBaseApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

DbApp.answer_yesBool

Default: False

Answer yes to any prompts.

DbApp.config_fileUnicode

Default: ''

Full path of a config file.

DbApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

DbApp.generate_configBool

Default: False

Generate default config file.

DbApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

DbApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

DbApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

DbApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

DbApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

DbApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

UpdateApp.answer_yesBool

Default: False

Answer yes to any prompts.

UpdateApp.config_fileUnicode

Default: ''

Full path of a config file.

UpdateApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

UpdateApp.generate_configBool

Default: False

Generate default config file.

UpdateApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

UpdateApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

UpdateApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

UpdateApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

UpdateApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

UpdateApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

UpdateApp.validateBool

Default: True

whether to validate metadata after updating it

GenerateConfigApp.answer_yesBool

Default: False

Answer yes to any prompts.

GenerateConfigApp.config_fileUnicode

Default: ''

Full path of a config file.

GenerateConfigApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

GenerateConfigApp.filenameUnicode

Default: 'nbgrader_config.py'

The name of the configuration file to generate.

GenerateConfigApp.generate_configBool

Default: False

Generate default config file.

GenerateConfigApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

GenerateConfigApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

GenerateConfigApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

GenerateConfigApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

GenerateConfigApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

GenerateConfigApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

GenerateSolutionApp.answer_yesBool

Default: False

Answer yes to any prompts.

GenerateSolutionApp.config_fileUnicode

Default: ''

Full path of a config file.

GenerateSolutionApp.config_file_nameUnicode

Default: ''

Specify a config file to load.

GenerateSolutionApp.generate_configBool

Default: False

Generate default config file.

GenerateSolutionApp.log_datefmtUnicode

Default: '%Y-%m-%d %H:%M:%S'

The date format used by logging formatters for %(asctime)s

GenerateSolutionApp.log_formatUnicode

Default: '[%(name)s]%(highlevel)s %(message)s'

The Logging format template

GenerateSolutionApp.log_levelany of 0``|``10``|``20``|``30``|``40``|``50``|’DEBUG’|’INFO’|’WARN’|’ERROR’|’CRITICAL’``

Default: 30

Set the log level by value or name.

GenerateSolutionApp.logfileUnicode

Default: ''

Name of the logfile to log to. By default, log output is not written to any file.

GenerateSolutionApp.show_configBool

Default: False

Instead of starting the Application, dump configuration to stdout

GenerateSolutionApp.show_config_jsonBool

Default: False

Instead of starting the Application, dump configuration to stdout (as JSON)

ExportPlugin.assignmentList

Default: []

list of assignments to export

ExportPlugin.studentList

Default: []

list of students to export

ExportPlugin.toUnicode

Default: ''

destination to export to

CsvExportPlugin.assignmentList

Default: []

list of assignments to export

CsvExportPlugin.studentList

Default: []

list of students to export

CsvExportPlugin.toUnicode

Default: ''

destination to export to

ExtractorPlugin.forceBool

Default: False

Force overwrite of existing files.

ExtractorPlugin.zip_extList

Default: ['.zip', '.gz']

List of valid archive (zip) filename extensions to extract. Any archive (zip) files with an extension not in this list are copied to the extracted_directory.

FileNameCollectorPlugin.named_regexpUnicode

Default: ''

This regular expression is applied to each submission filename and MUST be supplied by the instructor. This regular expression MUST provide the (?P<student_id>…) and (?P<file_id>…) named group expressions. Optionally this regular expression can also provide the (?P<first_name>…), (?P<last_name>…), (?P<email>…), and (?P<timestamp>…) named group expressions. For example if the filename is:

ps1_bitdiddle_attempt_2016-01-30-15-00-00_problem1.ipynb

then this named_regexp could be:

“.*_(?P<student_id>w+)_attempt_(?P<timestamp>[0-9-]+)_(?P<file_id>w+)”

For named group regular expression examples see https://docs.python.org/3/howto/regex.html

FileNameCollectorPlugin.valid_extList

Default: ['.ipynb']

List of valid submission filename extensions to collect. Any submitted file with an extension not in this list is skipped.

LateSubmissionPlugin.penalty_methodany of 'none'``|’zero’``

Default: 'none'

The method for assigning late submission penalties:: ‘none’: do nothing (no penalty assigned) ‘zero’: assign an overall score of zero (penalty = score)

NbConvertBase.default_languageUnicode

Default: 'ipython'

Deprecated default highlight language as of 5.0, please use language_info metadata instead

NbConvertBase.display_data_priorityList

Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

An ordered list of preferred output type, the first encountered will usually be used when converting discarding the others.

Preprocessor.default_languageUnicode

Default: 'ipython'

Deprecated default highlight language as of 5.0, please use language_info metadata instead

Preprocessor.display_data_priorityList

Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

An ordered list of preferred output type, the first encountered will usually be used when converting discarding the others.

Preprocessor.enabledBool

Default: False

No description

NbGraderPreprocessor.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

AssignLatePenalties.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

AssignLatePenalties.plugin_classType

Default: 'nbgrader.plugins.latesubmission.LateSubmissionPlugin'

The plugin class for assigning the late penalty for each notebook.

IncludeHeaderFooter.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

IncludeHeaderFooter.footerUnicode

Default: ''

Path to footer notebook, relative to the root of the course directory

IncludeHeaderFooter.headerUnicode

Default: ''

Path to header notebook, relative to the root of the course directory

LockCells.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

LockCells.lock_all_cellsBool

Default: False

Whether all assignment cells are locked (non-deletable and non-editable)

LockCells.lock_grade_cellsBool

Default: True

Whether grade cells are locked (non-deletable)

LockCells.lock_readonly_cellsBool

Default: True

Whether readonly cells are locked (non-deletable and non-editable)

LockCells.lock_solution_cellsBool

Default: True

Whether solution cells are locked (non-deletable and non-editable)

ClearSolutions.begin_solution_delimeterUnicode

Default: 'BEGIN SOLUTION'

The delimiter marking the beginning of a solution

ClearSolutions.code_stubDict

Default: {'python': '# YOUR CODE HERE\\nraise NotImplementedError()', '...

The code snippet that will replace code solutions

ClearSolutions.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

ClearSolutions.end_solution_delimeterUnicode

Default: 'END SOLUTION'

The delimiter marking the end of a solution

ClearSolutions.enforce_metadataBool

Default: True

Whether or not to complain if cells containing solutions regions are not marked as solution cells. WARNING: this will potentially cause things to break if you are using the full nbgrader pipeline. ONLY disable this option if you are only ever planning to use nbgrader assign.

ClearSolutions.text_stubUnicode

Default: 'YOUR ANSWER HERE'

The text snippet that will replace written solutions

SaveAutoGrades.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

ComputeChecksums.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

SaveCells.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

OverwriteCells.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

CheckCellMetadata.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

NotebookClient.allow_error_namesList

Default: []

List of error names which won’t stop the execution. Use this if the allow_errors option it too general and you want to allow only specific kinds of errors.

NotebookClient.allow_errorsBool

Default: False

If False (default), when a cell raises an error the execution is stopped and a CellExecutionError is raised, except if the error name is in allow_error_names. If True, execution errors are ignored and the execution is continued until the end of the notebook. Output from exceptions is included in the cell output in both cases.

NotebookClient.display_data_priorityList

Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

An ordered list of preferred output type, the first encountered will usually be used when converting discarding the others.

NotebookClient.error_on_timeoutDict

Default: None

If a cell execution was interrupted after a timeout, don’t wait for the execute_reply from the kernel (e.g. KeyboardInterrupt error). Instead, return an execute_reply with the given error, which should be of the following form:

{
    'ename': str,  # Exception name, as a string
    'evalue': str,  # Exception value, as a string
    'traceback': list(str),  # traceback frames, as strings
}

NotebookClient.extra_argumentsList

Default: []

No description

NotebookClient.force_raise_errorsBool

Default: False

If False (default), errors from executing the notebook can be allowed with a raises-exception tag on a single cell, or the allow_errors or allow_error_names configurable options for all cells. An allowed error will be recorded in notebook output, and execution will continue. If an error occurs when it is not explicitly allowed, a CellExecutionError will be raised. If True, CellExecutionError will be raised for any error that occurs while executing the notebook. This overrides the allow_errors and allow_error_names options and the raises-exception cell tag.

NotebookClient.interrupt_on_timeoutBool

Default: False

If execution of a cell times out, interrupt the kernel and continue executing other cells rather than throwing an error and stopping.

NotebookClient.iopub_timeoutInt

Default: 4

The time to wait (in seconds) for IOPub output. This generally doesn’t need to be set, but on some slow networks (such as CI systems) the default timeout might not be long enough to get all messages.

NotebookClient.ipython_hist_fileUnicode

Default: ':memory:'

Path to file to use for SQLite history database for an IPython kernel.

The specific value :memory: (including the colon at both end but not the back ticks), avoids creating a history file. Otherwise, IPython will create a history file for each kernel.

When running kernels simultaneously (e.g. via multiprocessing) saving history a single SQLite file can result in database errors, so using :memory: is recommended in non-interactive contexts.

NotebookClient.kernel_manager_classType

Default: 'builtins.object'

The kernel manager class to use.

NotebookClient.kernel_nameUnicode

Default: ''

Name of kernel to use to execute the cells. If not set, use the kernel_spec embedded in the notebook.

NotebookClient.on_cell_completeCallable

Default: None

A callable which executes after a cell execution is complete. It is called even when a cell results in a failure. Called with kwargs cell and cell_index.

NotebookClient.on_cell_errorCallable

Default: None

A callable which executes when a cell execution results in an error. This is executed even if errors are suppressed with cell_allows_errors. Called with kwargs cell`, ``cell_index and execute_reply.

NotebookClient.on_cell_executeCallable

Default: None

A callable which executes just before a code cell is executed. Called with kwargs cell and cell_index.

NotebookClient.on_cell_executedCallable

Default: None

A callable which executes just after a code cell is executed, whether or not it results in an error. Called with kwargs cell, cell_index and execute_reply.

NotebookClient.on_cell_startCallable

Default: None

A callable which executes before a cell is executed and before non-executing cells are skipped. Called with kwargs cell and cell_index.

NotebookClient.on_notebook_completeCallable

Default: None

A callable which executes after the kernel is cleaned up. Called with kwargs notebook.

NotebookClient.on_notebook_errorCallable

Default: None

A callable which executes when the notebook encounters an error. Called with kwargs notebook.

NotebookClient.on_notebook_startCallable

Default: None

A callable which executes after the kernel manager and kernel client are setup, and cells are about to execute. Called with kwargs notebook.

NotebookClient.raise_on_iopub_timeoutBool

Default: False

If False (default), then the kernel will continue waiting for iopub messages until it receives a kernel idle message, or until a timeout occurs, at which point the currently executing cell will be skipped. If True, then an error will be raised after the first timeout. This option generally does not need to be used, but may be useful in contexts where there is the possibility of executing notebooks with memory-consuming infinite loops.

NotebookClient.record_timingBool

Default: True

If True (default), then the execution timings of each cell will be stored in the metadata of the notebook.

NotebookClient.shell_timeout_intervalInt

Default: 5

The time to wait (in seconds) for Shell output before retrying. This generally doesn’t need to be set, but if one needs to check for dead kernels at a faster rate this can help.

NotebookClient.shutdown_kernelany of 'graceful'``|’immediate’``

Default: 'graceful'

If graceful (default), then the kernel is given time to clean up after executing all cells, e.g., to execute its atexit hooks. If immediate, then the kernel is signaled to immediately terminate.

NotebookClient.skip_cells_with_tagUnicode

Default: 'skip-execution'

Name of the cell tag to use to denote a cell that should be skipped.

NotebookClient.startup_timeoutInt

Default: 60

The time to wait (in seconds) for the kernel to start. If kernel startup takes longer, a RuntimeError is raised.

NotebookClient.store_widget_stateBool

Default: True

If True (default), then the state of the Jupyter widgets created at the kernel will be stored in the metadata of the notebook.

NotebookClient.timeoutInt

Default: None

The time to wait (in seconds) for output from executions. If a cell execution takes longer, a TimeoutError is raised.

None or -1 will disable the timeout. If timeout_func is set, it overrides timeout.

NotebookClient.timeout_funcAny

Default: None

A callable which, when given the cell source as input, returns the time to wait (in seconds) for output from cell executions. If a cell execution takes longer, a TimeoutError is raised.

Returning None or -1 will disable the timeout for the cell. Not setting timeout_func will cause the client to default to using the timeout trait for all cells. The timeout_func trait overrides timeout if it is not None.

ExecutePreprocessor.allow_error_namesList

Default: []

List of error names which won’t stop the execution. Use this if the allow_errors option it too general and you want to allow only specific kinds of errors.

ExecutePreprocessor.allow_errorsBool

Default: False

If False (default), when a cell raises an error the execution is stopped and a CellExecutionError is raised, except if the error name is in allow_error_names. If True, execution errors are ignored and the execution is continued until the end of the notebook. Output from exceptions is included in the cell output in both cases.

ExecutePreprocessor.default_languageUnicode

Default: 'ipython'

Deprecated default highlight language as of 5.0, please use language_info metadata instead

ExecutePreprocessor.display_data_priorityList

Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

An ordered list of preferred output type, the first encountered will usually be used when converting discarding the others.

ExecutePreprocessor.enabledBool

Default: False

No description

ExecutePreprocessor.error_on_timeoutDict

Default: None

If a cell execution was interrupted after a timeout, don’t wait for the execute_reply from the kernel (e.g. KeyboardInterrupt error). Instead, return an execute_reply with the given error, which should be of the following form:

{
    'ename': str,  # Exception name, as a string
    'evalue': str,  # Exception value, as a string
    'traceback': list(str),  # traceback frames, as strings
}

ExecutePreprocessor.extra_argumentsList

Default: []

No description

ExecutePreprocessor.force_raise_errorsBool

Default: False

If False (default), errors from executing the notebook can be allowed with a raises-exception tag on a single cell, or the allow_errors or allow_error_names configurable options for all cells. An allowed error will be recorded in notebook output, and execution will continue. If an error occurs when it is not explicitly allowed, a CellExecutionError will be raised. If True, CellExecutionError will be raised for any error that occurs while executing the notebook. This overrides the allow_errors and allow_error_names options and the raises-exception cell tag.

ExecutePreprocessor.interrupt_on_timeoutBool

Default: False

If execution of a cell times out, interrupt the kernel and continue executing other cells rather than throwing an error and stopping.

ExecutePreprocessor.iopub_timeoutInt

Default: 4

The time to wait (in seconds) for IOPub output. This generally doesn’t need to be set, but on some slow networks (such as CI systems) the default timeout might not be long enough to get all messages.

ExecutePreprocessor.ipython_hist_fileUnicode

Default: ':memory:'

Path to file to use for SQLite history database for an IPython kernel.

The specific value :memory: (including the colon at both end but not the back ticks), avoids creating a history file. Otherwise, IPython will create a history file for each kernel.

When running kernels simultaneously (e.g. via multiprocessing) saving history a single SQLite file can result in database errors, so using :memory: is recommended in non-interactive contexts.

ExecutePreprocessor.kernel_manager_classType

Default: 'builtins.object'

The kernel manager class to use.

ExecutePreprocessor.kernel_nameUnicode

Default: ''

Name of kernel to use to execute the cells. If not set, use the kernel_spec embedded in the notebook.

ExecutePreprocessor.on_cell_completeCallable

Default: None

A callable which executes after a cell execution is complete. It is called even when a cell results in a failure. Called with kwargs cell and cell_index.

ExecutePreprocessor.on_cell_errorCallable

Default: None

A callable which executes when a cell execution results in an error. This is executed even if errors are suppressed with cell_allows_errors. Called with kwargs cell`, ``cell_index and execute_reply.

ExecutePreprocessor.on_cell_executeCallable

Default: None

A callable which executes just before a code cell is executed. Called with kwargs cell and cell_index.

ExecutePreprocessor.on_cell_executedCallable

Default: None

A callable which executes just after a code cell is executed, whether or not it results in an error. Called with kwargs cell, cell_index and execute_reply.

ExecutePreprocessor.on_cell_startCallable

Default: None

A callable which executes before a cell is executed and before non-executing cells are skipped. Called with kwargs cell and cell_index.

ExecutePreprocessor.on_notebook_completeCallable

Default: None

A callable which executes after the kernel is cleaned up. Called with kwargs notebook.

ExecutePreprocessor.on_notebook_errorCallable

Default: None

A callable which executes when the notebook encounters an error. Called with kwargs notebook.

ExecutePreprocessor.on_notebook_startCallable

Default: None

A callable which executes after the kernel manager and kernel client are setup, and cells are about to execute. Called with kwargs notebook.

ExecutePreprocessor.raise_on_iopub_timeoutBool

Default: False

If False (default), then the kernel will continue waiting for iopub messages until it receives a kernel idle message, or until a timeout occurs, at which point the currently executing cell will be skipped. If True, then an error will be raised after the first timeout. This option generally does not need to be used, but may be useful in contexts where there is the possibility of executing notebooks with memory-consuming infinite loops.

ExecutePreprocessor.record_timingBool

Default: True

If True (default), then the execution timings of each cell will be stored in the metadata of the notebook.

ExecutePreprocessor.shell_timeout_intervalInt

Default: 5

The time to wait (in seconds) for Shell output before retrying. This generally doesn’t need to be set, but if one needs to check for dead kernels at a faster rate this can help.

ExecutePreprocessor.shutdown_kernelany of 'graceful'``|’immediate’``

Default: 'graceful'

If graceful (default), then the kernel is given time to clean up after executing all cells, e.g., to execute its atexit hooks. If immediate, then the kernel is signaled to immediately terminate.

ExecutePreprocessor.skip_cells_with_tagUnicode

Default: 'skip-execution'

Name of the cell tag to use to denote a cell that should be skipped.

ExecutePreprocessor.startup_timeoutInt

Default: 60

The time to wait (in seconds) for the kernel to start. If kernel startup takes longer, a RuntimeError is raised.

ExecutePreprocessor.store_widget_stateBool

Default: True

If True (default), then the state of the Jupyter widgets created at the kernel will be stored in the metadata of the notebook.

ExecutePreprocessor.timeoutInt

Default: None

The time to wait (in seconds) for output from executions. If a cell execution takes longer, a TimeoutError is raised.

None or -1 will disable the timeout. If timeout_func is set, it overrides timeout.

ExecutePreprocessor.timeout_funcAny

Default: None

A callable which, when given the cell source as input, returns the time to wait (in seconds) for output from cell executions. If a cell execution takes longer, a TimeoutError is raised.

Returning None or -1 will disable the timeout for the cell. Not setting timeout_func will cause the client to default to using the timeout trait for all cells. The timeout_func trait overrides timeout if it is not None.

Execute.allow_error_namesList

Default: []

List of error names which won’t stop the execution. Use this if the allow_errors option it too general and you want to allow only specific kinds of errors.

Execute.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

Execute.execute_retriesInt

Default: 0

The number of times to try re-executing the notebook before throwing an error. Generally, this shouldn’t need to be set, but might be useful for CI environments when tests are flaky.

Execute.extra_argumentsList

Default: []

A list of extra arguments to pass to the kernel. For python kernels, this defaults to --HistoryManager.hist_file=:memory:. For other kernels this is just an empty list.

Execute.force_raise_errorsBool

Default: False

If False (default), errors from executing the notebook can be allowed with a raises-exception tag on a single cell, or the allow_errors or allow_error_names configurable options for all cells. An allowed error will be recorded in notebook output, and execution will continue. If an error occurs when it is not explicitly allowed, a CellExecutionError will be raised. If True, CellExecutionError will be raised for any error that occurs while executing the notebook. This overrides the allow_errors and allow_error_names options and the raises-exception cell tag.

Execute.iopub_timeoutInt

Default: 4

The time to wait (in seconds) for IOPub output. This generally doesn’t need to be set, but on some slow networks (such as CI systems) the default timeout might not be long enough to get all messages.

Execute.ipython_hist_fileUnicode

Default: ':memory:'

Path to file to use for SQLite history database for an IPython kernel.

The specific value :memory: (including the colon at both end but not the back ticks), avoids creating a history file. Otherwise, IPython will create a history file for each kernel.

When running kernels simultaneously (e.g. via multiprocessing) saving history a single SQLite file can result in database errors, so using :memory: is recommended in non-interactive contexts.

Execute.kernel_manager_classType

Default: 'builtins.object'

The kernel manager class to use.

Execute.kernel_nameUnicode

Default: ''

Name of kernel to use to execute the cells. If not set, use the kernel_spec embedded in the notebook.

Execute.on_cell_completeCallable

Default: None

A callable which executes after a cell execution is complete. It is called even when a cell results in a failure. Called with kwargs cell and cell_index.

Execute.on_cell_errorCallable

Default: None

A callable which executes when a cell execution results in an error. This is executed even if errors are suppressed with cell_allows_errors. Called with kwargs cell`, ``cell_index and execute_reply.

Execute.on_cell_executeCallable

Default: None

A callable which executes just before a code cell is executed. Called with kwargs cell and cell_index.

Execute.on_cell_startCallable

Default: None

A callable which executes before a cell is executed and before non-executing cells are skipped. Called with kwargs cell and cell_index.

Execute.on_notebook_completeCallable

Default: None

A callable which executes after the kernel is cleaned up. Called with kwargs notebook.

Execute.on_notebook_errorCallable

Default: None

A callable which executes when the notebook encounters an error. Called with kwargs notebook.

Execute.on_notebook_startCallable

Default: None

A callable which executes after the kernel manager and kernel client are setup, and cells are about to execute. Called with kwargs notebook.

Execute.record_timingBool

Default: True

If True (default), then the execution timings of each cell will be stored in the metadata of the notebook.

Execute.shell_timeout_intervalInt

Default: 5

The time to wait (in seconds) for Shell output before retrying. This generally doesn’t need to be set, but if one needs to check for dead kernels at a faster rate this can help.

Execute.shutdown_kernelany of 'graceful'``|’immediate’``

Default: 'graceful'

If graceful (default), then the kernel is given time to clean up after executing all cells, e.g., to execute its atexit hooks. If immediate, then the kernel is signaled to immediately terminate.

Execute.skip_cells_with_tagUnicode

Default: 'skip-execution'

Name of the cell tag to use to denote a cell that should be skipped.

Execute.startup_timeoutInt

Default: 60

The time to wait (in seconds) for the kernel to start. If kernel startup takes longer, a RuntimeError is raised.

Execute.store_widget_stateBool

Default: True

If True (default), then the state of the Jupyter widgets created at the kernel will be stored in the metadata of the notebook.

Execute.timeout_funcAny

Default: None

A callable which, when given the cell source as input, returns the time to wait (in seconds) for output from cell executions. If a cell execution takes longer, a TimeoutError is raised.

Returning None or -1 will disable the timeout for the cell. Not setting timeout_func will cause the client to default to using the timeout trait for all cells. The timeout_func trait overrides timeout if it is not None.

GetGrades.display_data_priorityList

Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

No description

GetGrades.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

ClearOutputPreprocessor.default_languageUnicode

Default: 'ipython'

Deprecated default highlight language as of 5.0, please use language_info metadata instead

ClearOutputPreprocessor.display_data_priorityList

Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

An ordered list of preferred output type, the first encountered will usually be used when converting discarding the others.

ClearOutputPreprocessor.enabledBool

Default: False

No description

ClearOutputPreprocessor.remove_metadata_fieldsSet

Default: {'collapsed', 'scrolled'}

No description

ClearOutput.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

ClearOutput.remove_metadata_fieldsSet

Default: {'collapsed', 'scrolled'}

No description

LimitOutput.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

LimitOutput.max_linesInt

Default: 1000

maximum number of lines of output (-1 means no limit)

LimitOutput.max_tracebackInt

Default: 100

maximum number of traceback lines (-1 means no limit)

DeduplicateIds.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

ClearHiddenTests.begin_test_delimeterUnicode

Default: 'BEGIN HIDDEN TESTS'

The delimiter marking the beginning of hidden tests cases

ClearHiddenTests.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

ClearHiddenTests.end_test_delimeterUnicode

Default: 'END HIDDEN TESTS'

The delimiter marking the end of hidden tests cases

ClearHiddenTests.enforce_metadataBool

Default: True

Whether or not to complain if cells containing hidden test regions are not marked as grade cells. WARNING: this will potentially cause things to break if you are using the full nbgrader pipeline. ONLY disable this option if you are only ever planning to use nbgrader assign.

ClearMarkScheme.begin_mark_scheme_delimeterUnicode

Default: 'BEGIN MARK SCHEME'

The delimiter marking the beginning of hidden tests cases

ClearMarkScheme.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

ClearMarkScheme.end_mark_scheme_delimeterUnicode

Default: 'END MARK SCHEME'

The delimiter marking the end of hidden tests cases

ClearMarkScheme.enforce_metadataBool

Default: True

Whether or not to complain if cells containing marking scheme regions are not marked as task cells. WARNING: this will potentially cause things to break if you are using the full nbgrader pipeline. ONLY disable this option if you are only ever planning to use nbgrader assign.

OverwriteKernelspec.enabledBool

Default: True

Whether to use this preprocessor when running nbgrader

Exchange.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

Exchange.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

Exchange.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeCollect.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeCollect.before_duedateBool

Default: False

Collect the last submission before due date or the last submission if no submission before due date.

ExchangeCollect.check_ownerBool

Default: True

Whether to cross-check the student_id with the UNIX-owner of the submitted directory.

ExchangeCollect.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeCollect.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeCollect.updateBool

Default: False

Update existing submissions with ones that have newer timestamps.

ExchangeFetchAssignment.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeFetchAssignment.replace_missing_filesBool

Default: False

Whether to replace missing files on fetch

ExchangeFetchAssignment.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeFetchAssignment.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeFetch.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeFetch.replace_missing_filesBool

Default: False

Whether to replace missing files on fetch

ExchangeFetch.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeFetch.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeFetchFeedback.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeFetchFeedback.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeFetchFeedback.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeList.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeList.cachedBool

Default: False

List assignments in submission cache.

ExchangeList.inboundBool

Default: False

List inbound files rather than outbound.

ExchangeList.removeBool

Default: False

Remove, rather than list files.

ExchangeList.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeList.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeReleaseAssignment.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeReleaseAssignment.forceBool

Default: False

Force overwrite existing files in the exchange.

ExchangeReleaseAssignment.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeReleaseAssignment.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeRelease.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeRelease.forceBool

Default: False

Force overwrite existing files in the exchange.

ExchangeRelease.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeRelease.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeReleaseFeedback.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeReleaseFeedback.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeReleaseFeedback.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

ExchangeSubmit.assignment_dirUnicode

Default: '.'

Local path for storing student assignments. Defaults to ‘.’ which is normally Jupyter’s notebook_dir.

ExchangeSubmit.strictBool

Default: False

Whether or not to submit the assignment if there are missing notebooks from the released assignment notebooks.

ExchangeSubmit.timestamp_formatUnicode

Default: '%Y-%m-%d %H:%M:%S.%f %Z'

Format string for timestamps

ExchangeSubmit.timezoneUnicode

Default: 'UTC'

Timezone for recording timestamps

BaseConverter.exporter_classType

Default: 'nbconvert.exporters.notebook.NotebookExporter'

No description

BaseConverter.forceBool

Default: False

Whether to overwrite existing assignments/submissions

BaseConverter.permissionsInt

Default: 0

Permissions to set on files output by nbgrader. The default is generally read-only (444), with the exception of nbgrader generate_assignment and nbgrader generate_feedback, in which case the user also has write permission.

BaseConverter.post_convert_hookAny

Default: None

An optional hook function that you can implement to do some work after converting. This function is called after the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

BaseConverter.pre_convert_hookAny

Default: None

An optional hook function that you can implement to do some bootstrapping work before converting. This function is called before the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

GenerateAssignment.create_assignmentBool

Default: True

Whether to create the assignment at runtime if it does not already exist.

GenerateAssignment.exporter_classType

Default: 'nbconvert.exporters.notebook.NotebookExporter'

No description

GenerateAssignment.forceBool

Default: False

Whether to overwrite existing assignments/submissions

GenerateAssignment.no_databaseBool

Default: False

Do not save information about the assignment into the database.

GenerateAssignment.permissionsInt

Default: 0

Permissions to set on files output by nbgrader. The default is generally read-only (444), with the exception of nbgrader generate_assignment and nbgrader generate_feedback, in which case the user also has write permission.

GenerateAssignment.post_convert_hookAny

Default: None

An optional hook function that you can implement to do some work after converting. This function is called after the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

GenerateAssignment.pre_convert_hookAny

Default: None

An optional hook function that you can implement to do some bootstrapping work before converting. This function is called before the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

GenerateAssignment.preprocessorsList

Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

No description

Assign.create_assignmentBool

Default: True

Whether to create the assignment at runtime if it does not already exist.

Assign.exporter_classType

Default: 'nbconvert.exporters.notebook.NotebookExporter'

No description

Assign.forceBool

Default: False

Whether to overwrite existing assignments/submissions

Assign.no_databaseBool

Default: False

Do not save information about the assignment into the database.

Assign.permissionsInt

Default: 0

Permissions to set on files output by nbgrader. The default is generally read-only (444), with the exception of nbgrader generate_assignment and nbgrader generate_feedback, in which case the user also has write permission.

Assign.post_convert_hookAny

Default: None

An optional hook function that you can implement to do some work after converting. This function is called after the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

Assign.pre_convert_hookAny

Default: None

An optional hook function that you can implement to do some bootstrapping work before converting. This function is called before the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

Assign.preprocessorsList

Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

No description

Autograde.autograde_preprocessorsList

Default: [<class 'nbgrader.preprocessors.execute.Execute'>, <class 'nb...

No description

Autograde.create_studentBool

Default: True

Whether to create the student at runtime if it does not already exist.

Autograde.exclude_overwritingDict

Default: {}

A dictionary with keys corresponding to assignment names and values being a list of filenames (relative to the assignment’s source directory) that should NOT be overwritten with the source version. This is to allow students to e.g. edit a python file and submit it alongside the notebooks in their assignment.

Autograde.exporter_classType

Default: 'nbconvert.exporters.notebook.NotebookExporter'

No description

Autograde.forceBool

Default: False

Whether to overwrite existing assignments/submissions

Autograde.permissionsInt

Default: 0

Permissions to set on files output by nbgrader. The default is generally read-only (444), with the exception of nbgrader generate_assignment and nbgrader generate_feedback, in which case the user also has write permission.

Autograde.post_convert_hookAny

Default: None

An optional hook function that you can implement to do some work after converting. This function is called after the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

Autograde.pre_convert_hookAny

Default: None

An optional hook function that you can implement to do some bootstrapping work before converting. This function is called before the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

Autograde.sanitize_preprocessorsList

Default: [<class 'nbgrader.preprocessors.clearoutput.ClearOutput'>, <c...

No description

GenerateFeedback.exporter_classType

Default: 'nbconvert.exporters.notebook.NotebookExporter'

No description

GenerateFeedback.forceBool

Default: False

Whether to overwrite existing assignments/submissions

GenerateFeedback.permissionsInt

Default: 0

Permissions to set on files output by nbgrader. The default is generally read-only (444), with the exception of nbgrader generate_assignment and nbgrader generate_feedback, in which case the user also has write permission.

GenerateFeedback.post_convert_hookAny

Default: None

An optional hook function that you can implement to do some work after converting. This function is called after the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

GenerateFeedback.pre_convert_hookAny

Default: None

An optional hook function that you can implement to do some bootstrapping work before converting. This function is called before the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

GenerateFeedback.preprocessorsList

Default: [<class 'nbgrader.preprocessors.getgrades.GetGrades'>, <class...

No description

Feedback.exporter_classType

Default: 'nbconvert.exporters.notebook.NotebookExporter'

No description

Feedback.forceBool

Default: False

Whether to overwrite existing assignments/submissions

Feedback.permissionsInt

Default: 0

Permissions to set on files output by nbgrader. The default is generally read-only (444), with the exception of nbgrader generate_assignment and nbgrader generate_feedback, in which case the user also has write permission.

Feedback.post_convert_hookAny

Default: None

An optional hook function that you can implement to do some work after converting. This function is called after the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

Feedback.pre_convert_hookAny

Default: None

An optional hook function that you can implement to do some bootstrapping work before converting. This function is called before the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

Feedback.preprocessorsList

Default: [<class 'nbgrader.preprocessors.getgrades.GetGrades'>, <class...

No description

GenerateSolution.create_assignmentBool

Default: True

Whether to create the assignment at runtime if it does not already exist.

GenerateSolution.exporter_classType

Default: 'nbconvert.exporters.notebook.NotebookExporter'

No description

GenerateSolution.forceBool

Default: False

Whether to overwrite existing assignments/submissions

GenerateSolution.permissionsInt

Default: 0

Permissions to set on files output by nbgrader. The default is generally read-only (444), with the exception of nbgrader generate_assignment and nbgrader generate_feedback, in which case the user also has write permission.

GenerateSolution.post_convert_hookAny

Default: None

An optional hook function that you can implement to do some work after converting. This function is called after the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

GenerateSolution.pre_convert_hookAny

Default: None

An optional hook function that you can implement to do some bootstrapping work before converting. This function is called before the notebooks are converted and should be used for specific converters such as Autograde, GenerateAssignment or GenerateFeedback.

It will be called as (all arguments are passed as keywords):

hook(assignment=assignment, student=student, notebooks=notebooks)

GenerateSolution.preprocessorsList

Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

No description

Command line options¶

Basic commands¶

nbgrader¶

A system for assigning and grading notebooks

Subcommands
===========
Subcommands are launched as `nbgrader cmd [args]`. For information on using
subcommand 'cmd', do: `nbgrader cmd -h`.

assign
    DEPRECATED, please use generate_assignment instead.
generate_assignment
    Create the student version of an assignment. Intended for use by
    instructors only.
autograde
    Autograde submitted assignments. Intended for use by instructors
    only.
formgrade
    Manually grade assignments (after autograding). Intended for use
    by instructors only.
feedback
    DEPRECATED: use generate_feedback instead.
generate_feedback
    Generate feedback (after autograding and manual grading).
    Intended for use by instructors only.
validate
    Validate a notebook in an assignment. Intended for use by
    instructors and students.
release
    DEPRECATED: use release_assignment instead.
release_assignment
    Release an assignment to students through the nbgrader exchange.
    Intended for use by instructors only.
release_feedback
    Release assignment feedback to students through the nbgrader exchange.
    Intended for use by instructors only.
collect
    Collect an assignment from students through the nbgrader exchange.
    Intended for use by instructors only.
zip_collect
    Collect assignment submissions from files and/or archives (zip
    files) manually downloaded from a LMS.
    Intended for use by instructors only.
fetch
    DEPRECATED: use fetch_assignment instead.
fetch_assignment
    Fetch an assignment from an instructor through the nbgrader exchange.
    Intended for use by students only.
fetch_feedback
    Fetch feedback for an assignment from an instructor through the nbgrader exchange.
    Intended for use by students only.
submit
    Submit an assignment to an instructor through the nbgrader exchange.
    Intended for use by students only.
list
    List inbound or outbound assignments in the nbgrader exchange.
    Intended for use by instructors and students.
extension
    Install and activate the "Create Assignment" notebook extension.
quickstart
    Create an example class files directory with an example
    config file and assignment.
export
    Export grades from the database to another format.
db
    Perform operations on the nbgrader database, such as adding,
    removing, importing, and listing assignments or students.
update
    Update nbgrader cell metadata to the most recent version.
generate_config
    Generates a default nbgrader_config.py file.
generate_solution
    Generates the solution for the given assignment.

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGraderApp(NbGrader) options
-----------------------------
--NbGraderApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGraderApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGraderApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGraderApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGraderApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGraderApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGraderApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGraderApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGraderApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGraderApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Authenticator(LoggingConfigurable) options
------------------------------------------
--Authenticator.plugin_class=<Type>
    A plugin for different authentication methods.
    Default: 'nbgrader.auth.base.NoAuthPlugin'

GenerateAssignmentApp(NbGrader) options
---------------------------------------
--GenerateAssignmentApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateAssignmentApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateAssignmentApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateAssignmentApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateAssignmentApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateAssignmentApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateAssignmentApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateAssignmentApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateAssignmentApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateAssignmentApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

AssignApp(GenerateAssignmentApp) options
----------------------------------------
--AssignApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--AssignApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--AssignApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--AssignApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--AssignApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--AssignApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--AssignApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--AssignApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--AssignApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--AssignApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

AutogradeApp(NbGrader) options
------------------------------
--AutogradeApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--AutogradeApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--AutogradeApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--AutogradeApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--AutogradeApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--AutogradeApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--AutogradeApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--AutogradeApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--AutogradeApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--AutogradeApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

FormgradeApp(NbGrader) options
------------------------------
--FormgradeApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--FormgradeApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--FormgradeApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--FormgradeApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--FormgradeApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--FormgradeApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--FormgradeApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--FormgradeApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--FormgradeApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--FormgradeApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

GenerateFeedbackApp(NbGrader) options
-------------------------------------
--GenerateFeedbackApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateFeedbackApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateFeedbackApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateFeedbackApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateFeedbackApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateFeedbackApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateFeedbackApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateFeedbackApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateFeedbackApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateFeedbackApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

FeedbackApp(GenerateFeedbackApp) options
----------------------------------------
--FeedbackApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--FeedbackApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--FeedbackApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--FeedbackApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--FeedbackApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--FeedbackApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--FeedbackApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--FeedbackApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--FeedbackApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--FeedbackApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ValidateApp(NbGrader) options
-----------------------------
--ValidateApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ValidateApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ValidateApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ValidateApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ValidateApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ValidateApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ValidateApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ValidateApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ValidateApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ValidateApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ReleaseAssignmentApp(NbGrader) options
--------------------------------------
--ReleaseAssignmentApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ReleaseAssignmentApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ReleaseAssignmentApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ReleaseAssignmentApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ReleaseAssignmentApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ReleaseAssignmentApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ReleaseAssignmentApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ReleaseAssignmentApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ReleaseAssignmentApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ReleaseAssignmentApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ReleaseApp(ReleaseAssignmentApp) options
----------------------------------------
--ReleaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ReleaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ReleaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ReleaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ReleaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ReleaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ReleaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ReleaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ReleaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ReleaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ReleaseFeedbackApp(NbGrader) options
------------------------------------
--ReleaseFeedbackApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ReleaseFeedbackApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ReleaseFeedbackApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ReleaseFeedbackApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ReleaseFeedbackApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ReleaseFeedbackApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ReleaseFeedbackApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ReleaseFeedbackApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ReleaseFeedbackApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ReleaseFeedbackApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

CollectApp(NbGrader) options
----------------------------
--CollectApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--CollectApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--CollectApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--CollectApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--CollectApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--CollectApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--CollectApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--CollectApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--CollectApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--CollectApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ZipCollectApp(NbGrader) options
-------------------------------
--ZipCollectApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ZipCollectApp.archive_directory=<Unicode>
    The name of the directory that contains assignment submission files and/or
    archives (zip) files manually downloaded from a LMS. This corresponds to the
    `collect_step` variable in the `collect_structure` config option.
    Default: 'archive'
--ZipCollectApp.collect_directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the zip collect process. This MUST contain named keys for 'downloaded',
    'assignment_id', and 'collect_step'.
    Default: '{downloaded}/{assignment_id}/{collect_step}'
--ZipCollectApp.collector_plugin=<Type>
    The plugin class for processing the submitted file names after they have
    been extracted into the `extracted_directory`.
    Default: 'nbgrader.plugins.zipcollect.FileNameCollectorPlugin'
--ZipCollectApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ZipCollectApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ZipCollectApp.downloaded_directory=<Unicode>
    The main directory that corresponds to the `downloaded` variable in the
    `collect_structure` config option.
    Default: 'downloaded'
--ZipCollectApp.extracted_directory=<Unicode>
    The name of the directory that contains assignment submission files
    extracted or copied from the `archive_directory`. This corresponds to the
    `collect_step` variable in the `collect_structure` config option.
    Default: 'extracted'
--ZipCollectApp.extractor_plugin=<Type>
    The plugin class for extracting the archive files in the
    `archive_directory`.
    Default: 'nbgrader.plugins.zipcollect.ExtractorPlugin'
--ZipCollectApp.force=<Bool>
    Force overwrite of existing files.
    Default: False
--ZipCollectApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ZipCollectApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ZipCollectApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ZipCollectApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ZipCollectApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ZipCollectApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ZipCollectApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False
--ZipCollectApp.strict=<Bool>
    Skip submitted notebooks with invalid names.
    Default: False

FetchAssignmentApp(NbGrader) options
------------------------------------
--FetchAssignmentApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--FetchAssignmentApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--FetchAssignmentApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--FetchAssignmentApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--FetchAssignmentApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--FetchAssignmentApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--FetchAssignmentApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--FetchAssignmentApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--FetchAssignmentApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--FetchAssignmentApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

FetchApp(FetchAssignmentApp) options
------------------------------------
--FetchApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--FetchApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--FetchApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--FetchApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--FetchApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--FetchApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--FetchApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--FetchApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--FetchApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--FetchApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

FetchFeedbackApp(NbGrader) options
----------------------------------
--FetchFeedbackApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--FetchFeedbackApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--FetchFeedbackApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--FetchFeedbackApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--FetchFeedbackApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--FetchFeedbackApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--FetchFeedbackApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--FetchFeedbackApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--FetchFeedbackApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--FetchFeedbackApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

SubmitApp(NbGrader) options
---------------------------
--SubmitApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--SubmitApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--SubmitApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--SubmitApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--SubmitApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--SubmitApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--SubmitApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--SubmitApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--SubmitApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--SubmitApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ListApp(NbGrader) options
-------------------------
--ListApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ListApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ListApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ListApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ListApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ListApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ListApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ListApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ListApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ListApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExtensionApp(NbGrader) options
------------------------------
--ExtensionApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ExtensionApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ExtensionApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ExtensionApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ExtensionApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ExtensionApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ExtensionApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ExtensionApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ExtensionApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ExtensionApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

QuickStartApp(NbGrader) options
-------------------------------
--QuickStartApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--QuickStartApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--QuickStartApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--QuickStartApp.force=<Bool>
    Whether to overwrite existing files
    Default: False
--QuickStartApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--QuickStartApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--QuickStartApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--QuickStartApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--QuickStartApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--QuickStartApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--QuickStartApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExportApp(NbGrader) options
---------------------------
--ExportApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ExportApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ExportApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ExportApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ExportApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ExportApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ExportApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ExportApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ExportApp.plugin_class=<Type>
    The plugin class for exporting the grades.
    Default: 'nbgrader.plugins.export.CsvExportPlugin'
--ExportApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ExportApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbApp(DbBaseApp) options
------------------------
--DbApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

UpdateApp(NbGrader) options
---------------------------
--UpdateApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--UpdateApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--UpdateApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--UpdateApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--UpdateApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--UpdateApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--UpdateApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--UpdateApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--UpdateApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--UpdateApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False
--UpdateApp.validate=<Bool>
    whether to validate metadata after updating it
    Default: True

GenerateConfigApp(NbGrader) options
-----------------------------------
--GenerateConfigApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateConfigApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateConfigApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateConfigApp.filename=<Unicode>
    The name of the configuration file to generate.
    Default: 'nbgrader_config.py'
--GenerateConfigApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateConfigApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateConfigApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateConfigApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateConfigApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateConfigApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateConfigApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

GenerateSolutionApp(NbGrader) options
-------------------------------------
--GenerateSolutionApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateSolutionApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateSolutionApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateSolutionApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateSolutionApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateSolutionApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateSolutionApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateSolutionApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateSolutionApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateSolutionApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExportPlugin(BasePlugin) options
--------------------------------
--ExportPlugin.assignment=<list-item-1>...
    list of assignments to export
    Default: []
--ExportPlugin.student=<list-item-1>...
    list of students to export
    Default: []
--ExportPlugin.to=<Unicode>
    destination to export to
    Default: ''

CsvExportPlugin(ExportPlugin) options
-------------------------------------
--CsvExportPlugin.assignment=<list-item-1>...
    list of assignments to export
    Default: []
--CsvExportPlugin.student=<list-item-1>...
    list of students to export
    Default: []
--CsvExportPlugin.to=<Unicode>
    destination to export to
    Default: ''

ExtractorPlugin(BasePlugin) options
-----------------------------------
--ExtractorPlugin.force=<Bool>
    Force overwrite of existing files.
    Default: False
--ExtractorPlugin.zip_ext=<list-item-1>...
    List of valid archive (zip) filename extensions to extract. Any archive
    (zip) files with an extension not in this list are copied to the
    `extracted_directory`.
    Default: ['.zip', '.gz']

FileNameCollectorPlugin(BasePlugin) options
-------------------------------------------
--FileNameCollectorPlugin.named_regexp=<Unicode>
    This regular expression is applied to each submission filename and MUST be
    supplied by the instructor. This regular expression MUST provide the
    `(?P<student_id>...)` and `(?P<file_id>...)` named group expressions.
    Optionally this regular expression can also provide the
    `(?P<first_name>...)`, `(?P<last_name>...)`, `(?P<email>...)`, and
    `(?P<timestamp>...)` named group expressions. For example if the filename
    is:
        `ps1_bitdiddle_attempt_2016-01-30-15-00-00_problem1.ipynb`
    then this `named_regexp` could be:
    ".*_(?P<student_id>\w+)_attempt_(?P<timestamp>[0-9\-]+)_(?P<file_id>\w+)"
    For named group regular expression examples see
    https://docs.python.org/3/howto/regex.html
    Default: ''
--FileNameCollectorPlugin.valid_ext=<list-item-1>...
    List of valid submission filename extensions to collect. Any submitted file
    with an extension not in this list is skipped.
    Default: ['.ipynb']

LateSubmissionPlugin(BasePlugin) options
----------------------------------------
--LateSubmissionPlugin.penalty_method=<Enum>
    The method for assigning late submission penalties:
        'none': do nothing (no penalty assigned)
        'zero': assign an overall score of zero (penalty = score)
    Choices: any of ['none', 'zero']
    Default: 'none'

NbConvertBase(LoggingConfigurable) options
------------------------------------------
--NbConvertBase.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--NbConvertBase.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

Preprocessor(NbConvertBase) options
-----------------------------------
--Preprocessor.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--Preprocessor.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--Preprocessor.enabled=<Bool>
    Default: False

NbGraderPreprocessor(Preprocessor) options
------------------------------------------
--NbGraderPreprocessor.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

AssignLatePenalties(NbGraderPreprocessor) options
-------------------------------------------------
--AssignLatePenalties.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--AssignLatePenalties.plugin_class=<Type>
    The plugin class for assigning the late penalty for each notebook.
    Default: 'nbgrader.plugins.latesubmission.LateSubmissionPlugin'

IncludeHeaderFooter(NbGraderPreprocessor) options
-------------------------------------------------
--IncludeHeaderFooter.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--IncludeHeaderFooter.footer=<Unicode>
    Path to footer notebook, relative to the root of the course directory
    Default: ''
--IncludeHeaderFooter.header=<Unicode>
    Path to header notebook, relative to the root of the course directory
    Default: ''

LockCells(NbGraderPreprocessor) options
---------------------------------------
--LockCells.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--LockCells.lock_all_cells=<Bool>
    Whether all assignment cells are locked (non-deletable and non-editable)
    Default: False
--LockCells.lock_grade_cells=<Bool>
    Whether grade cells are locked (non-deletable)
    Default: True
--LockCells.lock_readonly_cells=<Bool>
    Whether readonly cells are locked (non-deletable and non-editable)
    Default: True
--LockCells.lock_solution_cells=<Bool>
    Whether solution cells are locked (non-deletable and non-editable)
    Default: True

ClearSolutions(NbGraderPreprocessor) options
--------------------------------------------
--ClearSolutions.begin_solution_delimeter=<Unicode>
    The delimiter marking the beginning of a solution
    Default: 'BEGIN SOLUTION'
--ClearSolutions.code_stub=<key-1>=<value-1>...
    The code snippet that will replace code solutions
    Default: {'python': '# YOUR CODE HERE\nraise NotImplementedError()', '...
--ClearSolutions.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearSolutions.end_solution_delimeter=<Unicode>
    The delimiter marking the end of a solution
    Default: 'END SOLUTION'
--ClearSolutions.enforce_metadata=<Bool>
    Whether or not to complain if cells containing solutions regions are not
    marked as solution cells. WARNING: this will potentially cause things to
    break if you are using the full nbgrader pipeline. ONLY disable this option
    if you are only ever planning to use nbgrader assign.
    Default: True
--ClearSolutions.text_stub=<Unicode>
    The text snippet that will replace written solutions
    Default: 'YOUR ANSWER HERE'

SaveAutoGrades(NbGraderPreprocessor) options
--------------------------------------------
--SaveAutoGrades.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

ComputeChecksums(NbGraderPreprocessor) options
----------------------------------------------
--ComputeChecksums.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

SaveCells(NbGraderPreprocessor) options
---------------------------------------
--SaveCells.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

OverwriteCells(NbGraderPreprocessor) options
--------------------------------------------
--OverwriteCells.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

CheckCellMetadata(NbGraderPreprocessor) options
-----------------------------------------------
--CheckCellMetadata.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

NotebookClient(LoggingConfigurable) options
-------------------------------------------
--NotebookClient.allow_error_names=<list-item-1>...
    List of error names which won't stop the execution. Use this if the
    ``allow_errors`` option it too general and you want to allow only specific
    kinds of errors.
    Default: []
--NotebookClient.allow_errors=<Bool>
    If ``False`` (default), when a cell raises an error the execution is stopped
    and a ``CellExecutionError`` is raised, except if the error name is in
    ``allow_error_names``. If ``True``, execution errors are ignored and the
    execution is continued until the end of the notebook. Output from exceptions
    is included in the cell output in both cases.
    Default: False
--NotebookClient.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--NotebookClient.error_on_timeout=<key-1>=<value-1>...
    If a cell execution was interrupted after a timeout, don't wait for the
    execute_reply from the kernel (e.g. KeyboardInterrupt error). Instead,
    return an execute_reply with the given error, which should be of the
    following form::
        {
            'ename': str,  # Exception name, as a string
            'evalue': str,  # Exception value, as a string
            'traceback': list(str),  # traceback frames, as strings
        }
    Default: None
--NotebookClient.extra_arguments=<list-item-1>...
    Default: []
--NotebookClient.force_raise_errors=<Bool>
    If False (default), errors from executing the notebook can be allowed with a
    ``raises-exception`` tag on a single cell, or the ``allow_errors`` or
    ``allow_error_names`` configurable options for all cells. An allowed error
    will be recorded in notebook output, and execution will continue. If an
    error occurs when it is not explicitly allowed, a ``CellExecutionError``
    will be raised. If True, ``CellExecutionError`` will be raised for any error
    that occurs while executing the notebook. This overrides the
    ``allow_errors`` and ``allow_error_names`` options and the ``raises-
    exception`` cell tag.
    Default: False
--NotebookClient.interrupt_on_timeout=<Bool>
    If execution of a cell times out, interrupt the kernel and continue
    executing other cells rather than throwing an error and stopping.
    Default: False
--NotebookClient.iopub_timeout=<Int>
    The time to wait (in seconds) for IOPub output. This generally doesn't need
    to be set, but on some slow networks (such as CI systems) the default
    timeout might not be long enough to get all messages.
    Default: 4
--NotebookClient.ipython_hist_file=<Unicode>
    Path to file to use for SQLite history database for an IPython kernel.
            The specific value ``:memory:`` (including the colon
            at both end but not the back ticks), avoids creating a history file. Otherwise, IPython
            will create a history file for each kernel.
            When running kernels simultaneously (e.g. via multiprocessing) saving history a single
            SQLite file can result in database errors, so using ``:memory:`` is recommended in
            non-interactive contexts.
    Default: ':memory:'
--NotebookClient.kernel_manager_class=<Type>
    The kernel manager class to use.
    Default: 'builtins.object'
--NotebookClient.kernel_name=<Unicode>
    Name of kernel to use to execute the cells. If not set, use the kernel_spec
    embedded in the notebook.
    Default: ''
--NotebookClient.on_cell_complete=<Callable>
    A callable which executes after a cell execution is complete. It is called
    even when a cell results in a failure. Called with kwargs ``cell`` and
    ``cell_index``.
    Default: None
--NotebookClient.on_cell_error=<Callable>
    A callable which executes when a cell execution results in an error. This is
    executed even if errors are suppressed with ``cell_allows_errors``. Called
    with kwargs ``cell`, ``cell_index`` and ``execute_reply``.
    Default: None
--NotebookClient.on_cell_execute=<Callable>
    A callable which executes just before a code cell is executed. Called with
    kwargs ``cell`` and ``cell_index``.
    Default: None
--NotebookClient.on_cell_executed=<Callable>
    A callable which executes just after a code cell is executed, whether or not
    it results in an error. Called with kwargs ``cell``, ``cell_index`` and
    ``execute_reply``.
    Default: None
--NotebookClient.on_cell_start=<Callable>
    A callable which executes before a cell is executed and before non-executing
    cells are skipped. Called with kwargs ``cell`` and ``cell_index``.
    Default: None
--NotebookClient.on_notebook_complete=<Callable>
    A callable which executes after the kernel is cleaned up. Called with kwargs
    ``notebook``.
    Default: None
--NotebookClient.on_notebook_error=<Callable>
    A callable which executes when the notebook encounters an error. Called with
    kwargs ``notebook``.
    Default: None
--NotebookClient.on_notebook_start=<Callable>
    A callable which executes after the kernel manager and kernel client are
    setup, and cells are about to execute. Called with kwargs ``notebook``.
    Default: None
--NotebookClient.raise_on_iopub_timeout=<Bool>
    If ``False`` (default), then the kernel will continue waiting for iopub
    messages until it receives a kernel idle message, or until a timeout occurs,
    at which point the currently executing cell will be skipped. If ``True``,
    then an error will be raised after the first timeout. This option generally
    does not need to be used, but may be useful in contexts where there is the
    possibility of executing notebooks with memory-consuming infinite loops.
    Default: False
--NotebookClient.record_timing=<Bool>
    If ``True`` (default), then the execution timings of each cell will be
    stored in the metadata of the notebook.
    Default: True
--NotebookClient.shell_timeout_interval=<Int>
    The time to wait (in seconds) for Shell output before retrying. This
    generally doesn't need to be set, but if one needs to check for dead kernels
    at a faster rate this can help.
    Default: 5
--NotebookClient.shutdown_kernel=<Enum>
    If ``graceful`` (default), then the kernel is given time to clean up after
    executing all cells, e.g., to execute its ``atexit`` hooks. If
    ``immediate``, then the kernel is signaled to immediately terminate.
    Choices: any of ['graceful', 'immediate']
    Default: 'graceful'
--NotebookClient.skip_cells_with_tag=<Unicode>
    Name of the cell tag to use to denote a cell that should be skipped.
    Default: 'skip-execution'
--NotebookClient.startup_timeout=<Int>
    The time to wait (in seconds) for the kernel to start. If kernel startup
    takes longer, a RuntimeError is raised.
    Default: 60
--NotebookClient.store_widget_state=<Bool>
    If ``True`` (default), then the state of the Jupyter widgets created at the
    kernel will be stored in the metadata of the notebook.
    Default: True
--NotebookClient.timeout=<Int>
    The time to wait (in seconds) for output from executions. If a cell
    execution takes longer, a TimeoutError is raised.
    ``None`` or ``-1`` will disable the timeout. If ``timeout_func`` is set, it
    overrides ``timeout``.
    Default: None
--NotebookClient.timeout_func=<Any>
    A callable which, when given the cell source as input, returns the time to
    wait (in seconds) for output from cell executions. If a cell execution takes
    longer, a TimeoutError is raised.
    Returning ``None`` or ``-1`` will disable the timeout for the cell. Not
    setting ``timeout_func`` will cause the client to default to using the
    ``timeout`` trait for all cells. The ``timeout_func`` trait overrides
    ``timeout`` if it is not ``None``.
    Default: None

ExecutePreprocessor(Preprocessor, NotebookClient) options
---------------------------------------------------------
--ExecutePreprocessor.allow_error_names=<list-item-1>...
    List of error names which won't stop the execution. Use this if the
    ``allow_errors`` option it too general and you want to allow only specific
    kinds of errors.
    Default: []
--ExecutePreprocessor.allow_errors=<Bool>
    If ``False`` (default), when a cell raises an error the execution is stopped
    and a ``CellExecutionError`` is raised, except if the error name is in
    ``allow_error_names``. If ``True``, execution errors are ignored and the
    execution is continued until the end of the notebook. Output from exceptions
    is included in the cell output in both cases.
    Default: False
--ExecutePreprocessor.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--ExecutePreprocessor.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--ExecutePreprocessor.enabled=<Bool>
    Default: False
--ExecutePreprocessor.error_on_timeout=<key-1>=<value-1>...
    If a cell execution was interrupted after a timeout, don't wait for the
    execute_reply from the kernel (e.g. KeyboardInterrupt error). Instead,
    return an execute_reply with the given error, which should be of the
    following form::
        {
            'ename': str,  # Exception name, as a string
            'evalue': str,  # Exception value, as a string
            'traceback': list(str),  # traceback frames, as strings
        }
    Default: None
--ExecutePreprocessor.extra_arguments=<list-item-1>...
    Default: []
--ExecutePreprocessor.force_raise_errors=<Bool>
    If False (default), errors from executing the notebook can be allowed with a
    ``raises-exception`` tag on a single cell, or the ``allow_errors`` or
    ``allow_error_names`` configurable options for all cells. An allowed error
    will be recorded in notebook output, and execution will continue. If an
    error occurs when it is not explicitly allowed, a ``CellExecutionError``
    will be raised. If True, ``CellExecutionError`` will be raised for any error
    that occurs while executing the notebook. This overrides the
    ``allow_errors`` and ``allow_error_names`` options and the ``raises-
    exception`` cell tag.
    Default: False
--ExecutePreprocessor.interrupt_on_timeout=<Bool>
    If execution of a cell times out, interrupt the kernel and continue
    executing other cells rather than throwing an error and stopping.
    Default: False
--ExecutePreprocessor.iopub_timeout=<Int>
    The time to wait (in seconds) for IOPub output. This generally doesn't need
    to be set, but on some slow networks (such as CI systems) the default
    timeout might not be long enough to get all messages.
    Default: 4
--ExecutePreprocessor.ipython_hist_file=<Unicode>
    Path to file to use for SQLite history database for an IPython kernel.
            The specific value ``:memory:`` (including the colon
            at both end but not the back ticks), avoids creating a history file. Otherwise, IPython
            will create a history file for each kernel.
            When running kernels simultaneously (e.g. via multiprocessing) saving history a single
            SQLite file can result in database errors, so using ``:memory:`` is recommended in
            non-interactive contexts.
    Default: ':memory:'
--ExecutePreprocessor.kernel_manager_class=<Type>
    The kernel manager class to use.
    Default: 'builtins.object'
--ExecutePreprocessor.kernel_name=<Unicode>
    Name of kernel to use to execute the cells. If not set, use the kernel_spec
    embedded in the notebook.
    Default: ''
--ExecutePreprocessor.on_cell_complete=<Callable>
    A callable which executes after a cell execution is complete. It is called
    even when a cell results in a failure. Called with kwargs ``cell`` and
    ``cell_index``.
    Default: None
--ExecutePreprocessor.on_cell_error=<Callable>
    A callable which executes when a cell execution results in an error. This is
    executed even if errors are suppressed with ``cell_allows_errors``. Called
    with kwargs ``cell`, ``cell_index`` and ``execute_reply``.
    Default: None
--ExecutePreprocessor.on_cell_execute=<Callable>
    A callable which executes just before a code cell is executed. Called with
    kwargs ``cell`` and ``cell_index``.
    Default: None
--ExecutePreprocessor.on_cell_executed=<Callable>
    A callable which executes just after a code cell is executed, whether or not
    it results in an error. Called with kwargs ``cell``, ``cell_index`` and
    ``execute_reply``.
    Default: None
--ExecutePreprocessor.on_cell_start=<Callable>
    A callable which executes before a cell is executed and before non-executing
    cells are skipped. Called with kwargs ``cell`` and ``cell_index``.
    Default: None
--ExecutePreprocessor.on_notebook_complete=<Callable>
    A callable which executes after the kernel is cleaned up. Called with kwargs
    ``notebook``.
    Default: None
--ExecutePreprocessor.on_notebook_error=<Callable>
    A callable which executes when the notebook encounters an error. Called with
    kwargs ``notebook``.
    Default: None
--ExecutePreprocessor.on_notebook_start=<Callable>
    A callable which executes after the kernel manager and kernel client are
    setup, and cells are about to execute. Called with kwargs ``notebook``.
    Default: None
--ExecutePreprocessor.raise_on_iopub_timeout=<Bool>
    If ``False`` (default), then the kernel will continue waiting for iopub
    messages until it receives a kernel idle message, or until a timeout occurs,
    at which point the currently executing cell will be skipped. If ``True``,
    then an error will be raised after the first timeout. This option generally
    does not need to be used, but may be useful in contexts where there is the
    possibility of executing notebooks with memory-consuming infinite loops.
    Default: False
--ExecutePreprocessor.record_timing=<Bool>
    If ``True`` (default), then the execution timings of each cell will be
    stored in the metadata of the notebook.
    Default: True
--ExecutePreprocessor.shell_timeout_interval=<Int>
    The time to wait (in seconds) for Shell output before retrying. This
    generally doesn't need to be set, but if one needs to check for dead kernels
    at a faster rate this can help.
    Default: 5
--ExecutePreprocessor.shutdown_kernel=<Enum>
    If ``graceful`` (default), then the kernel is given time to clean up after
    executing all cells, e.g., to execute its ``atexit`` hooks. If
    ``immediate``, then the kernel is signaled to immediately terminate.
    Choices: any of ['graceful', 'immediate']
    Default: 'graceful'
--ExecutePreprocessor.skip_cells_with_tag=<Unicode>
    Name of the cell tag to use to denote a cell that should be skipped.
    Default: 'skip-execution'
--ExecutePreprocessor.startup_timeout=<Int>
    The time to wait (in seconds) for the kernel to start. If kernel startup
    takes longer, a RuntimeError is raised.
    Default: 60
--ExecutePreprocessor.store_widget_state=<Bool>
    If ``True`` (default), then the state of the Jupyter widgets created at the
    kernel will be stored in the metadata of the notebook.
    Default: True
--ExecutePreprocessor.timeout=<Int>
    The time to wait (in seconds) for output from executions. If a cell
    execution takes longer, a TimeoutError is raised.
    ``None`` or ``-1`` will disable the timeout. If ``timeout_func`` is set, it
    overrides ``timeout``.
    Default: None
--ExecutePreprocessor.timeout_func=<Any>
    A callable which, when given the cell source as input, returns the time to
    wait (in seconds) for output from cell executions. If a cell execution takes
    longer, a TimeoutError is raised.
    Returning ``None`` or ``-1`` will disable the timeout for the cell. Not
    setting ``timeout_func`` will cause the client to default to using the
    ``timeout`` trait for all cells. The ``timeout_func`` trait overrides
    ``timeout`` if it is not ``None``.
    Default: None

Execute(NbGraderPreprocessor, ExecutePreprocessor) options
----------------------------------------------------------
--Execute.allow_error_names=<list-item-1>...
    List of error names which won't stop the execution. Use this if the
    ``allow_errors`` option it too general and you want to allow only specific
    kinds of errors.
    Default: []
--Execute.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--Execute.execute_retries=<Int>
    The number of times to try re-executing the notebook before throwing an
    error. Generally, this shouldn't need to be set, but might be useful for CI
    environments when tests are flaky.
    Default: 0
--Execute.extra_arguments=<list-item-1>...
    A list of extra arguments to pass to the kernel. For python kernels, this
    defaults to ``--HistoryManager.hist_file=:memory:``. For other kernels this
    is just an empty list.
    Default: []
--Execute.force_raise_errors=<Bool>
    If False (default), errors from executing the notebook can be allowed with a
    ``raises-exception`` tag on a single cell, or the ``allow_errors`` or
    ``allow_error_names`` configurable options for all cells. An allowed error
    will be recorded in notebook output, and execution will continue. If an
    error occurs when it is not explicitly allowed, a ``CellExecutionError``
    will be raised. If True, ``CellExecutionError`` will be raised for any error
    that occurs while executing the notebook. This overrides the
    ``allow_errors`` and ``allow_error_names`` options and the ``raises-
    exception`` cell tag.
    Default: False
--Execute.iopub_timeout=<Int>
    The time to wait (in seconds) for IOPub output. This generally doesn't need
    to be set, but on some slow networks (such as CI systems) the default
    timeout might not be long enough to get all messages.
    Default: 4
--Execute.ipython_hist_file=<Unicode>
    Path to file to use for SQLite history database for an IPython kernel.
            The specific value ``:memory:`` (including the colon
            at both end but not the back ticks), avoids creating a history file. Otherwise, IPython
            will create a history file for each kernel.
            When running kernels simultaneously (e.g. via multiprocessing) saving history a single
            SQLite file can result in database errors, so using ``:memory:`` is recommended in
            non-interactive contexts.
    Default: ':memory:'
--Execute.kernel_manager_class=<Type>
    The kernel manager class to use.
    Default: 'builtins.object'
--Execute.kernel_name=<Unicode>
    Name of kernel to use to execute the cells. If not set, use the kernel_spec
    embedded in the notebook.
    Default: ''
--Execute.on_cell_complete=<Callable>
    A callable which executes after a cell execution is complete. It is called
    even when a cell results in a failure. Called with kwargs ``cell`` and
    ``cell_index``.
    Default: None
--Execute.on_cell_error=<Callable>
    A callable which executes when a cell execution results in an error. This is
    executed even if errors are suppressed with ``cell_allows_errors``. Called
    with kwargs ``cell`, ``cell_index`` and ``execute_reply``.
    Default: None
--Execute.on_cell_execute=<Callable>
    A callable which executes just before a code cell is executed. Called with
    kwargs ``cell`` and ``cell_index``.
    Default: None
--Execute.on_cell_start=<Callable>
    A callable which executes before a cell is executed and before non-executing
    cells are skipped. Called with kwargs ``cell`` and ``cell_index``.
    Default: None
--Execute.on_notebook_complete=<Callable>
    A callable which executes after the kernel is cleaned up. Called with kwargs
    ``notebook``.
    Default: None
--Execute.on_notebook_error=<Callable>
    A callable which executes when the notebook encounters an error. Called with
    kwargs ``notebook``.
    Default: None
--Execute.on_notebook_start=<Callable>
    A callable which executes after the kernel manager and kernel client are
    setup, and cells are about to execute. Called with kwargs ``notebook``.
    Default: None
--Execute.record_timing=<Bool>
    If ``True`` (default), then the execution timings of each cell will be
    stored in the metadata of the notebook.
    Default: True
--Execute.shell_timeout_interval=<Int>
    The time to wait (in seconds) for Shell output before retrying. This
    generally doesn't need to be set, but if one needs to check for dead kernels
    at a faster rate this can help.
    Default: 5
--Execute.shutdown_kernel=<Enum>
    If ``graceful`` (default), then the kernel is given time to clean up after
    executing all cells, e.g., to execute its ``atexit`` hooks. If
    ``immediate``, then the kernel is signaled to immediately terminate.
    Choices: any of ['graceful', 'immediate']
    Default: 'graceful'
--Execute.skip_cells_with_tag=<Unicode>
    Name of the cell tag to use to denote a cell that should be skipped.
    Default: 'skip-execution'
--Execute.startup_timeout=<Int>
    The time to wait (in seconds) for the kernel to start. If kernel startup
    takes longer, a RuntimeError is raised.
    Default: 60
--Execute.store_widget_state=<Bool>
    If ``True`` (default), then the state of the Jupyter widgets created at the
    kernel will be stored in the metadata of the notebook.
    Default: True
--Execute.timeout_func=<Any>
    A callable which, when given the cell source as input, returns the time to
    wait (in seconds) for output from cell executions. If a cell execution takes
    longer, a TimeoutError is raised.
    Returning ``None`` or ``-1`` will disable the timeout for the cell. Not
    setting ``timeout_func`` will cause the client to default to using the
    ``timeout`` trait for all cells. The ``timeout_func`` trait overrides
    ``timeout`` if it is not ``None``.
    Default: None

GetGrades(NbGraderPreprocessor) options
---------------------------------------
--GetGrades.display_data_priority=<list-item-1>...
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--GetGrades.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

ClearOutputPreprocessor(Preprocessor) options
---------------------------------------------
--ClearOutputPreprocessor.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--ClearOutputPreprocessor.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--ClearOutputPreprocessor.enabled=<Bool>
    Default: False
--ClearOutputPreprocessor.remove_metadata_fields=<set-item-1>...
    Default: {'collapsed', 'scrolled'}

ClearOutput(NbGraderPreprocessor, ClearOutputPreprocessor) options
------------------------------------------------------------------
--ClearOutput.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearOutput.remove_metadata_fields=<set-item-1>...
    Default: {'collapsed', 'scrolled'}

LimitOutput(NbGraderPreprocessor) options
-----------------------------------------
--LimitOutput.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--LimitOutput.max_lines=<Int>
    maximum number of lines of output (-1 means no limit)
    Default: 1000
--LimitOutput.max_traceback=<Int>
    maximum number of traceback lines (-1 means no limit)
    Default: 100

DeduplicateIds(NbGraderPreprocessor) options
--------------------------------------------
--DeduplicateIds.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

ClearHiddenTests(NbGraderPreprocessor) options
----------------------------------------------
--ClearHiddenTests.begin_test_delimeter=<Unicode>
    The delimiter marking the beginning of hidden tests cases
    Default: 'BEGIN HIDDEN TESTS'
--ClearHiddenTests.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearHiddenTests.end_test_delimeter=<Unicode>
    The delimiter marking the end of hidden tests cases
    Default: 'END HIDDEN TESTS'
--ClearHiddenTests.enforce_metadata=<Bool>
    Whether or not to complain if cells containing hidden test regions are not
    marked as grade cells. WARNING: this will potentially cause things to break
    if you are using the full nbgrader pipeline. ONLY disable this option if you
    are only ever planning to use nbgrader assign.
    Default: True

ClearMarkScheme(NbGraderPreprocessor) options
---------------------------------------------
--ClearMarkScheme.begin_mark_scheme_delimeter=<Unicode>
    The delimiter marking the beginning of hidden tests cases
    Default: 'BEGIN MARK SCHEME'
--ClearMarkScheme.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearMarkScheme.end_mark_scheme_delimeter=<Unicode>
    The delimiter marking the end of hidden tests cases
    Default: 'END MARK SCHEME'
--ClearMarkScheme.enforce_metadata=<Bool>
    Whether or not to complain if cells containing marking scheme regions are
    not marked as task cells. WARNING: this will potentially cause things to
    break if you are using the full nbgrader pipeline. ONLY disable this option
    if you are only ever planning to use nbgrader assign.
    Default: True

OverwriteKernelspec(NbGraderPreprocessor) options
-------------------------------------------------
--OverwriteKernelspec.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeCollect(Exchange) options
---------------------------------
--ExchangeCollect.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeCollect.before_duedate=<Bool>
    Collect the last submission before due date or the last submission if no
    submission before due date.
    Default: False
--ExchangeCollect.check_owner=<Bool>
    Whether to cross-check the student_id with the UNIX-owner of the submitted
    directory.
    Default: True
--ExchangeCollect.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeCollect.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
--ExchangeCollect.update=<Bool>
    Update existing submissions with ones that have newer timestamps.
    Default: False

ExchangeFetchAssignment(Exchange) options
-----------------------------------------
--ExchangeFetchAssignment.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetchAssignment.replace_missing_files=<Bool>
    Whether to replace missing files on fetch
    Default: False
--ExchangeFetchAssignment.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetchAssignment.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeFetch(ExchangeFetchAssignment) options
----------------------------------------------
--ExchangeFetch.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetch.replace_missing_files=<Bool>
    Whether to replace missing files on fetch
    Default: False
--ExchangeFetch.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetch.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeFetchFeedback(Exchange) options
---------------------------------------
--ExchangeFetchFeedback.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetchFeedback.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetchFeedback.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeList(Exchange) options
------------------------------
--ExchangeList.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeList.cached=<Bool>
    List assignments in submission cache.
    Default: False
--ExchangeList.inbound=<Bool>
    List inbound files rather than outbound.
    Default: False
--ExchangeList.remove=<Bool>
    Remove, rather than list files.
    Default: False
--ExchangeList.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeList.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeReleaseAssignment(Exchange) options
-------------------------------------------
--ExchangeReleaseAssignment.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeReleaseAssignment.force=<Bool>
    Force overwrite existing files in the exchange.
    Default: False
--ExchangeReleaseAssignment.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeReleaseAssignment.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeRelease(ExchangeReleaseAssignment) options
--------------------------------------------------
--ExchangeRelease.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeRelease.force=<Bool>
    Force overwrite existing files in the exchange.
    Default: False
--ExchangeRelease.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeRelease.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeReleaseFeedback(Exchange) options
-----------------------------------------
--ExchangeReleaseFeedback.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeReleaseFeedback.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeReleaseFeedback.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeSubmit(Exchange) options
--------------------------------
--ExchangeSubmit.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeSubmit.strict=<Bool>
    Whether or not to submit the assignment if there are missing notebooks from
    the released assignment notebooks.
    Default: False
--ExchangeSubmit.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeSubmit.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

BaseConverter(LoggingConfigurable) options
------------------------------------------
--BaseConverter.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--BaseConverter.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--BaseConverter.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--BaseConverter.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--BaseConverter.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None

GenerateAssignment(BaseConverter) options
-----------------------------------------
--GenerateAssignment.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--GenerateAssignment.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateAssignment.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateAssignment.no_database=<Bool>
    Do not save information about the assignment into the database.
    Default: False
--GenerateAssignment.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateAssignment.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateAssignment.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateAssignment.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Assign(GenerateAssignment) options
----------------------------------
--Assign.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--Assign.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Assign.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--Assign.no_database=<Bool>
    Do not save information about the assignment into the database.
    Default: False
--Assign.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--Assign.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Assign.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Assign.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Autograde(BaseConverter) options
--------------------------------
--Autograde.autograde_preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.execute.Execute'>, <class 'nb...
--Autograde.create_student=<Bool>
    Whether to create the student at runtime if it does not already exist.
    Default: True
--Autograde.exclude_overwriting=<key-1>=<value-1>...
    A dictionary with keys corresponding to assignment names and values being a
    list of filenames (relative to the assignment's source directory) that
    should NOT be overwritten with the source version. This is to allow students
    to e.g. edit a python file and submit it alongside the notebooks in their
    assignment.
    Default: {}
--Autograde.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Autograde.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--Autograde.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--Autograde.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Autograde.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Autograde.sanitize_preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.clearoutput.ClearOutput'>, <c...

GenerateFeedback(BaseConverter) options
---------------------------------------
--GenerateFeedback.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateFeedback.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateFeedback.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateFeedback.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateFeedback.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateFeedback.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.getgrades.GetGrades'>, <class...

Feedback(GenerateFeedback) options
----------------------------------
--Feedback.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Feedback.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--Feedback.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--Feedback.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Feedback.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Feedback.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.getgrades.GetGrades'>, <class...

GenerateSolution(BaseConverter) options
---------------------------------------
--GenerateSolution.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--GenerateSolution.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateSolution.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateSolution.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateSolution.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateSolution.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateSolution.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Examples
--------

    The nbgrader application is a system for assigning and grading notebooks.
            Each subcommand of this program corresponds to a different step in the
            grading process. In order to facilitate the grading pipeline, nbgrader
            places some constraints on how the assignments must be structured. By
            default, the directory structure for the assignments must look like this:

                {nbgrader_step}/{student_id}/{assignment_id}/{notebook_id}.ipynb

            where 'nbgrader_step' is the step in the nbgrader pipeline, 'student_id'
            is the ID of the student, 'assignment_id' is the name of the assignment,
            and 'notebook_id' is the name of the notebook (excluding the extension).
            For example, when running `nbgrader autograde "Problem Set 1"`, the
            autograder will first look for all notebooks for all students in the
            following directories:

                submitted/*/Problem Set 1/*.ipynb

            and it will write the autograded notebooks to the corresponding directory
            and filename for each notebook and each student:

                autograded/{student_id}/Problem Set 1/{notebook_id}.ipynb

            These variables, as well as the overall directory structure, can be
            configured through the `NbGrader` class (run `nbgrader --help-all`
            to see these options).

            For more details on how each of the subcommands work, please see the help
            for that command (e.g. `nbgrader generate_assignment --help-all`).

nbgrader generate config¶

Generates a default nbgrader_config.py file

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

GenerateConfigApp(NbGrader) options
-----------------------------------
--GenerateConfigApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateConfigApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateConfigApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateConfigApp.filename=<Unicode>
    The name of the configuration file to generate.
    Default: 'nbgrader_config.py'
--GenerateConfigApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateConfigApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateConfigApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateConfigApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateConfigApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateConfigApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateConfigApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Authenticator(LoggingConfigurable) options
------------------------------------------
--Authenticator.plugin_class=<Type>
    A plugin for different authentication methods.
    Default: 'nbgrader.auth.base.NoAuthPlugin'

ExportPlugin(BasePlugin) options
--------------------------------
--ExportPlugin.assignment=<list-item-1>...
    list of assignments to export
    Default: []
--ExportPlugin.student=<list-item-1>...
    list of students to export
    Default: []
--ExportPlugin.to=<Unicode>
    destination to export to
    Default: ''

CsvExportPlugin(ExportPlugin) options
-------------------------------------
--CsvExportPlugin.assignment=<list-item-1>...
    list of assignments to export
    Default: []
--CsvExportPlugin.student=<list-item-1>...
    list of students to export
    Default: []
--CsvExportPlugin.to=<Unicode>
    destination to export to
    Default: ''

ExtractorPlugin(BasePlugin) options
-----------------------------------
--ExtractorPlugin.force=<Bool>
    Force overwrite of existing files.
    Default: False
--ExtractorPlugin.zip_ext=<list-item-1>...
    List of valid archive (zip) filename extensions to extract. Any archive
    (zip) files with an extension not in this list are copied to the
    `extracted_directory`.
    Default: ['.zip', '.gz']

FileNameCollectorPlugin(BasePlugin) options
-------------------------------------------
--FileNameCollectorPlugin.named_regexp=<Unicode>
    This regular expression is applied to each submission filename and MUST be
    supplied by the instructor. This regular expression MUST provide the
    `(?P<student_id>...)` and `(?P<file_id>...)` named group expressions.
    Optionally this regular expression can also provide the
    `(?P<first_name>...)`, `(?P<last_name>...)`, `(?P<email>...)`, and
    `(?P<timestamp>...)` named group expressions. For example if the filename
    is:
        `ps1_bitdiddle_attempt_2016-01-30-15-00-00_problem1.ipynb`
    then this `named_regexp` could be:
    ".*_(?P<student_id>\w+)_attempt_(?P<timestamp>[0-9\-]+)_(?P<file_id>\w+)"
    For named group regular expression examples see
    https://docs.python.org/3/howto/regex.html
    Default: ''
--FileNameCollectorPlugin.valid_ext=<list-item-1>...
    List of valid submission filename extensions to collect. Any submitted file
    with an extension not in this list is skipped.
    Default: ['.ipynb']

LateSubmissionPlugin(BasePlugin) options
----------------------------------------
--LateSubmissionPlugin.penalty_method=<Enum>
    The method for assigning late submission penalties:
        'none': do nothing (no penalty assigned)
        'zero': assign an overall score of zero (penalty = score)
    Choices: any of ['none', 'zero']
    Default: 'none'

NbConvertBase(LoggingConfigurable) options
------------------------------------------
--NbConvertBase.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--NbConvertBase.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...

Preprocessor(NbConvertBase) options
-----------------------------------
--Preprocessor.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--Preprocessor.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--Preprocessor.enabled=<Bool>
    Default: False

NbGraderPreprocessor(Preprocessor) options
------------------------------------------
--NbGraderPreprocessor.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

AssignLatePenalties(NbGraderPreprocessor) options
-------------------------------------------------
--AssignLatePenalties.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--AssignLatePenalties.plugin_class=<Type>
    The plugin class for assigning the late penalty for each notebook.
    Default: 'nbgrader.plugins.latesubmission.LateSubmissionPlugin'

IncludeHeaderFooter(NbGraderPreprocessor) options
-------------------------------------------------
--IncludeHeaderFooter.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--IncludeHeaderFooter.footer=<Unicode>
    Path to footer notebook, relative to the root of the course directory
    Default: ''
--IncludeHeaderFooter.header=<Unicode>
    Path to header notebook, relative to the root of the course directory
    Default: ''

LockCells(NbGraderPreprocessor) options
---------------------------------------
--LockCells.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--LockCells.lock_all_cells=<Bool>
    Whether all assignment cells are locked (non-deletable and non-editable)
    Default: False
--LockCells.lock_grade_cells=<Bool>
    Whether grade cells are locked (non-deletable)
    Default: True
--LockCells.lock_readonly_cells=<Bool>
    Whether readonly cells are locked (non-deletable and non-editable)
    Default: True
--LockCells.lock_solution_cells=<Bool>
    Whether solution cells are locked (non-deletable and non-editable)
    Default: True

ClearSolutions(NbGraderPreprocessor) options
--------------------------------------------
--ClearSolutions.begin_solution_delimeter=<Unicode>
    The delimiter marking the beginning of a solution
    Default: 'BEGIN SOLUTION'
--ClearSolutions.code_stub=<key-1>=<value-1>...
    The code snippet that will replace code solutions
    Default: {'python': '# YOUR CODE HERE\nraise NotImplementedError()', '...
--ClearSolutions.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearSolutions.end_solution_delimeter=<Unicode>
    The delimiter marking the end of a solution
    Default: 'END SOLUTION'
--ClearSolutions.enforce_metadata=<Bool>
    Whether or not to complain if cells containing solutions regions are not
    marked as solution cells. WARNING: this will potentially cause things to
    break if you are using the full nbgrader pipeline. ONLY disable this option
    if you are only ever planning to use nbgrader assign.
    Default: True
--ClearSolutions.text_stub=<Unicode>
    The text snippet that will replace written solutions
    Default: 'YOUR ANSWER HERE'

SaveAutoGrades(NbGraderPreprocessor) options
--------------------------------------------
--SaveAutoGrades.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

ComputeChecksums(NbGraderPreprocessor) options
----------------------------------------------
--ComputeChecksums.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

SaveCells(NbGraderPreprocessor) options
---------------------------------------
--SaveCells.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

OverwriteCells(NbGraderPreprocessor) options
--------------------------------------------
--OverwriteCells.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

CheckCellMetadata(NbGraderPreprocessor) options
-----------------------------------------------
--CheckCellMetadata.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

NotebookClient(LoggingConfigurable) options
-------------------------------------------
--NotebookClient.allow_error_names=<list-item-1>...
    List of error names which won't stop the execution. Use this if the
    ``allow_errors`` option it too general and you want to allow only specific
    kinds of errors.
    Default: []
--NotebookClient.allow_errors=<Bool>
    If ``False`` (default), when a cell raises an error the execution is stopped
    and a ``CellExecutionError`` is raised, except if the error name is in
    ``allow_error_names``. If ``True``, execution errors are ignored and the
    execution is continued until the end of the notebook. Output from exceptions
    is included in the cell output in both cases.
    Default: False
--NotebookClient.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--NotebookClient.error_on_timeout=<key-1>=<value-1>...
    If a cell execution was interrupted after a timeout, don't wait for the
    execute_reply from the kernel (e.g. KeyboardInterrupt error). Instead,
    return an execute_reply with the given error, which should be of the
    following form::
        {
            'ename': str,  # Exception name, as a string
            'evalue': str,  # Exception value, as a string
            'traceback': list(str),  # traceback frames, as strings
        }
    Default: None
--NotebookClient.extra_arguments=<list-item-1>...
    Default: []
--NotebookClient.force_raise_errors=<Bool>
    If False (default), errors from executing the notebook can be allowed with a
    ``raises-exception`` tag on a single cell, or the ``allow_errors`` or
    ``allow_error_names`` configurable options for all cells. An allowed error
    will be recorded in notebook output, and execution will continue. If an
    error occurs when it is not explicitly allowed, a ``CellExecutionError``
    will be raised. If True, ``CellExecutionError`` will be raised for any error
    that occurs while executing the notebook. This overrides the
    ``allow_errors`` and ``allow_error_names`` options and the ``raises-
    exception`` cell tag.
    Default: False
--NotebookClient.interrupt_on_timeout=<Bool>
    If execution of a cell times out, interrupt the kernel and continue
    executing other cells rather than throwing an error and stopping.
    Default: False
--NotebookClient.iopub_timeout=<Int>
    The time to wait (in seconds) for IOPub output. This generally doesn't need
    to be set, but on some slow networks (such as CI systems) the default
    timeout might not be long enough to get all messages.
    Default: 4
--NotebookClient.ipython_hist_file=<Unicode>
    Path to file to use for SQLite history database for an IPython kernel.
            The specific value ``:memory:`` (including the colon
            at both end but not the back ticks), avoids creating a history file. Otherwise, IPython
            will create a history file for each kernel.
            When running kernels simultaneously (e.g. via multiprocessing) saving history a single
            SQLite file can result in database errors, so using ``:memory:`` is recommended in
            non-interactive contexts.
    Default: ':memory:'
--NotebookClient.kernel_manager_class=<Type>
    The kernel manager class to use.
    Default: 'builtins.object'
--NotebookClient.kernel_name=<Unicode>
    Name of kernel to use to execute the cells. If not set, use the kernel_spec
    embedded in the notebook.
    Default: ''
--NotebookClient.on_cell_complete=<Callable>
    A callable which executes after a cell execution is complete. It is called
    even when a cell results in a failure. Called with kwargs ``cell`` and
    ``cell_index``.
    Default: None
--NotebookClient.on_cell_error=<Callable>
    A callable which executes when a cell execution results in an error. This is
    executed even if errors are suppressed with ``cell_allows_errors``. Called
    with kwargs ``cell`, ``cell_index`` and ``execute_reply``.
    Default: None
--NotebookClient.on_cell_execute=<Callable>
    A callable which executes just before a code cell is executed. Called with
    kwargs ``cell`` and ``cell_index``.
    Default: None
--NotebookClient.on_cell_executed=<Callable>
    A callable which executes just after a code cell is executed, whether or not
    it results in an error. Called with kwargs ``cell``, ``cell_index`` and
    ``execute_reply``.
    Default: None
--NotebookClient.on_cell_start=<Callable>
    A callable which executes before a cell is executed and before non-executing
    cells are skipped. Called with kwargs ``cell`` and ``cell_index``.
    Default: None
--NotebookClient.on_notebook_complete=<Callable>
    A callable which executes after the kernel is cleaned up. Called with kwargs
    ``notebook``.
    Default: None
--NotebookClient.on_notebook_error=<Callable>
    A callable which executes when the notebook encounters an error. Called with
    kwargs ``notebook``.
    Default: None
--NotebookClient.on_notebook_start=<Callable>
    A callable which executes after the kernel manager and kernel client are
    setup, and cells are about to execute. Called with kwargs ``notebook``.
    Default: None
--NotebookClient.raise_on_iopub_timeout=<Bool>
    If ``False`` (default), then the kernel will continue waiting for iopub
    messages until it receives a kernel idle message, or until a timeout occurs,
    at which point the currently executing cell will be skipped. If ``True``,
    then an error will be raised after the first timeout. This option generally
    does not need to be used, but may be useful in contexts where there is the
    possibility of executing notebooks with memory-consuming infinite loops.
    Default: False
--NotebookClient.record_timing=<Bool>
    If ``True`` (default), then the execution timings of each cell will be
    stored in the metadata of the notebook.
    Default: True
--NotebookClient.shell_timeout_interval=<Int>
    The time to wait (in seconds) for Shell output before retrying. This
    generally doesn't need to be set, but if one needs to check for dead kernels
    at a faster rate this can help.
    Default: 5
--NotebookClient.shutdown_kernel=<Enum>
    If ``graceful`` (default), then the kernel is given time to clean up after
    executing all cells, e.g., to execute its ``atexit`` hooks. If
    ``immediate``, then the kernel is signaled to immediately terminate.
    Choices: any of ['graceful', 'immediate']
    Default: 'graceful'
--NotebookClient.skip_cells_with_tag=<Unicode>
    Name of the cell tag to use to denote a cell that should be skipped.
    Default: 'skip-execution'
--NotebookClient.startup_timeout=<Int>
    The time to wait (in seconds) for the kernel to start. If kernel startup
    takes longer, a RuntimeError is raised.
    Default: 60
--NotebookClient.store_widget_state=<Bool>
    If ``True`` (default), then the state of the Jupyter widgets created at the
    kernel will be stored in the metadata of the notebook.
    Default: True
--NotebookClient.timeout=<Int>
    The time to wait (in seconds) for output from executions. If a cell
    execution takes longer, a TimeoutError is raised.
    ``None`` or ``-1`` will disable the timeout. If ``timeout_func`` is set, it
    overrides ``timeout``.
    Default: None
--NotebookClient.timeout_func=<Any>
    A callable which, when given the cell source as input, returns the time to
    wait (in seconds) for output from cell executions. If a cell execution takes
    longer, a TimeoutError is raised.
    Returning ``None`` or ``-1`` will disable the timeout for the cell. Not
    setting ``timeout_func`` will cause the client to default to using the
    ``timeout`` trait for all cells. The ``timeout_func`` trait overrides
    ``timeout`` if it is not ``None``.
    Default: None

ExecutePreprocessor(Preprocessor, NotebookClient) options
---------------------------------------------------------
--ExecutePreprocessor.allow_error_names=<list-item-1>...
    List of error names which won't stop the execution. Use this if the
    ``allow_errors`` option it too general and you want to allow only specific
    kinds of errors.
    Default: []
--ExecutePreprocessor.allow_errors=<Bool>
    If ``False`` (default), when a cell raises an error the execution is stopped
    and a ``CellExecutionError`` is raised, except if the error name is in
    ``allow_error_names``. If ``True``, execution errors are ignored and the
    execution is continued until the end of the notebook. Output from exceptions
    is included in the cell output in both cases.
    Default: False
--ExecutePreprocessor.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--ExecutePreprocessor.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--ExecutePreprocessor.enabled=<Bool>
    Default: False
--ExecutePreprocessor.error_on_timeout=<key-1>=<value-1>...
    If a cell execution was interrupted after a timeout, don't wait for the
    execute_reply from the kernel (e.g. KeyboardInterrupt error). Instead,
    return an execute_reply with the given error, which should be of the
    following form::
        {
            'ename': str,  # Exception name, as a string
            'evalue': str,  # Exception value, as a string
            'traceback': list(str),  # traceback frames, as strings
        }
    Default: None
--ExecutePreprocessor.extra_arguments=<list-item-1>...
    Default: []
--ExecutePreprocessor.force_raise_errors=<Bool>
    If False (default), errors from executing the notebook can be allowed with a
    ``raises-exception`` tag on a single cell, or the ``allow_errors`` or
    ``allow_error_names`` configurable options for all cells. An allowed error
    will be recorded in notebook output, and execution will continue. If an
    error occurs when it is not explicitly allowed, a ``CellExecutionError``
    will be raised. If True, ``CellExecutionError`` will be raised for any error
    that occurs while executing the notebook. This overrides the
    ``allow_errors`` and ``allow_error_names`` options and the ``raises-
    exception`` cell tag.
    Default: False
--ExecutePreprocessor.interrupt_on_timeout=<Bool>
    If execution of a cell times out, interrupt the kernel and continue
    executing other cells rather than throwing an error and stopping.
    Default: False
--ExecutePreprocessor.iopub_timeout=<Int>
    The time to wait (in seconds) for IOPub output. This generally doesn't need
    to be set, but on some slow networks (such as CI systems) the default
    timeout might not be long enough to get all messages.
    Default: 4
--ExecutePreprocessor.ipython_hist_file=<Unicode>
    Path to file to use for SQLite history database for an IPython kernel.
            The specific value ``:memory:`` (including the colon
            at both end but not the back ticks), avoids creating a history file. Otherwise, IPython
            will create a history file for each kernel.
            When running kernels simultaneously (e.g. via multiprocessing) saving history a single
            SQLite file can result in database errors, so using ``:memory:`` is recommended in
            non-interactive contexts.
    Default: ':memory:'
--ExecutePreprocessor.kernel_manager_class=<Type>
    The kernel manager class to use.
    Default: 'builtins.object'
--ExecutePreprocessor.kernel_name=<Unicode>
    Name of kernel to use to execute the cells. If not set, use the kernel_spec
    embedded in the notebook.
    Default: ''
--ExecutePreprocessor.on_cell_complete=<Callable>
    A callable which executes after a cell execution is complete. It is called
    even when a cell results in a failure. Called with kwargs ``cell`` and
    ``cell_index``.
    Default: None
--ExecutePreprocessor.on_cell_error=<Callable>
    A callable which executes when a cell execution results in an error. This is
    executed even if errors are suppressed with ``cell_allows_errors``. Called
    with kwargs ``cell`, ``cell_index`` and ``execute_reply``.
    Default: None
--ExecutePreprocessor.on_cell_execute=<Callable>
    A callable which executes just before a code cell is executed. Called with
    kwargs ``cell`` and ``cell_index``.
    Default: None
--ExecutePreprocessor.on_cell_executed=<Callable>
    A callable which executes just after a code cell is executed, whether or not
    it results in an error. Called with kwargs ``cell``, ``cell_index`` and
    ``execute_reply``.
    Default: None
--ExecutePreprocessor.on_cell_start=<Callable>
    A callable which executes before a cell is executed and before non-executing
    cells are skipped. Called with kwargs ``cell`` and ``cell_index``.
    Default: None
--ExecutePreprocessor.on_notebook_complete=<Callable>
    A callable which executes after the kernel is cleaned up. Called with kwargs
    ``notebook``.
    Default: None
--ExecutePreprocessor.on_notebook_error=<Callable>
    A callable which executes when the notebook encounters an error. Called with
    kwargs ``notebook``.
    Default: None
--ExecutePreprocessor.on_notebook_start=<Callable>
    A callable which executes after the kernel manager and kernel client are
    setup, and cells are about to execute. Called with kwargs ``notebook``.
    Default: None
--ExecutePreprocessor.raise_on_iopub_timeout=<Bool>
    If ``False`` (default), then the kernel will continue waiting for iopub
    messages until it receives a kernel idle message, or until a timeout occurs,
    at which point the currently executing cell will be skipped. If ``True``,
    then an error will be raised after the first timeout. This option generally
    does not need to be used, but may be useful in contexts where there is the
    possibility of executing notebooks with memory-consuming infinite loops.
    Default: False
--ExecutePreprocessor.record_timing=<Bool>
    If ``True`` (default), then the execution timings of each cell will be
    stored in the metadata of the notebook.
    Default: True
--ExecutePreprocessor.shell_timeout_interval=<Int>
    The time to wait (in seconds) for Shell output before retrying. This
    generally doesn't need to be set, but if one needs to check for dead kernels
    at a faster rate this can help.
    Default: 5
--ExecutePreprocessor.shutdown_kernel=<Enum>
    If ``graceful`` (default), then the kernel is given time to clean up after
    executing all cells, e.g., to execute its ``atexit`` hooks. If
    ``immediate``, then the kernel is signaled to immediately terminate.
    Choices: any of ['graceful', 'immediate']
    Default: 'graceful'
--ExecutePreprocessor.skip_cells_with_tag=<Unicode>
    Name of the cell tag to use to denote a cell that should be skipped.
    Default: 'skip-execution'
--ExecutePreprocessor.startup_timeout=<Int>
    The time to wait (in seconds) for the kernel to start. If kernel startup
    takes longer, a RuntimeError is raised.
    Default: 60
--ExecutePreprocessor.store_widget_state=<Bool>
    If ``True`` (default), then the state of the Jupyter widgets created at the
    kernel will be stored in the metadata of the notebook.
    Default: True
--ExecutePreprocessor.timeout=<Int>
    The time to wait (in seconds) for output from executions. If a cell
    execution takes longer, a TimeoutError is raised.
    ``None`` or ``-1`` will disable the timeout. If ``timeout_func`` is set, it
    overrides ``timeout``.
    Default: None
--ExecutePreprocessor.timeout_func=<Any>
    A callable which, when given the cell source as input, returns the time to
    wait (in seconds) for output from cell executions. If a cell execution takes
    longer, a TimeoutError is raised.
    Returning ``None`` or ``-1`` will disable the timeout for the cell. Not
    setting ``timeout_func`` will cause the client to default to using the
    ``timeout`` trait for all cells. The ``timeout_func`` trait overrides
    ``timeout`` if it is not ``None``.
    Default: None

Execute(NbGraderPreprocessor, ExecutePreprocessor) options
----------------------------------------------------------
--Execute.allow_error_names=<list-item-1>...
    List of error names which won't stop the execution. Use this if the
    ``allow_errors`` option it too general and you want to allow only specific
    kinds of errors.
    Default: []
--Execute.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--Execute.execute_retries=<Int>
    The number of times to try re-executing the notebook before throwing an
    error. Generally, this shouldn't need to be set, but might be useful for CI
    environments when tests are flaky.
    Default: 0
--Execute.extra_arguments=<list-item-1>...
    A list of extra arguments to pass to the kernel. For python kernels, this
    defaults to ``--HistoryManager.hist_file=:memory:``. For other kernels this
    is just an empty list.
    Default: []
--Execute.force_raise_errors=<Bool>
    If False (default), errors from executing the notebook can be allowed with a
    ``raises-exception`` tag on a single cell, or the ``allow_errors`` or
    ``allow_error_names`` configurable options for all cells. An allowed error
    will be recorded in notebook output, and execution will continue. If an
    error occurs when it is not explicitly allowed, a ``CellExecutionError``
    will be raised. If True, ``CellExecutionError`` will be raised for any error
    that occurs while executing the notebook. This overrides the
    ``allow_errors`` and ``allow_error_names`` options and the ``raises-
    exception`` cell tag.
    Default: False
--Execute.iopub_timeout=<Int>
    The time to wait (in seconds) for IOPub output. This generally doesn't need
    to be set, but on some slow networks (such as CI systems) the default
    timeout might not be long enough to get all messages.
    Default: 4
--Execute.ipython_hist_file=<Unicode>
    Path to file to use for SQLite history database for an IPython kernel.
            The specific value ``:memory:`` (including the colon
            at both end but not the back ticks), avoids creating a history file. Otherwise, IPython
            will create a history file for each kernel.
            When running kernels simultaneously (e.g. via multiprocessing) saving history a single
            SQLite file can result in database errors, so using ``:memory:`` is recommended in
            non-interactive contexts.
    Default: ':memory:'
--Execute.kernel_manager_class=<Type>
    The kernel manager class to use.
    Default: 'builtins.object'
--Execute.kernel_name=<Unicode>
    Name of kernel to use to execute the cells. If not set, use the kernel_spec
    embedded in the notebook.
    Default: ''
--Execute.on_cell_complete=<Callable>
    A callable which executes after a cell execution is complete. It is called
    even when a cell results in a failure. Called with kwargs ``cell`` and
    ``cell_index``.
    Default: None
--Execute.on_cell_error=<Callable>
    A callable which executes when a cell execution results in an error. This is
    executed even if errors are suppressed with ``cell_allows_errors``. Called
    with kwargs ``cell`, ``cell_index`` and ``execute_reply``.
    Default: None
--Execute.on_cell_execute=<Callable>
    A callable which executes just before a code cell is executed. Called with
    kwargs ``cell`` and ``cell_index``.
    Default: None
--Execute.on_cell_start=<Callable>
    A callable which executes before a cell is executed and before non-executing
    cells are skipped. Called with kwargs ``cell`` and ``cell_index``.
    Default: None
--Execute.on_notebook_complete=<Callable>
    A callable which executes after the kernel is cleaned up. Called with kwargs
    ``notebook``.
    Default: None
--Execute.on_notebook_error=<Callable>
    A callable which executes when the notebook encounters an error. Called with
    kwargs ``notebook``.
    Default: None
--Execute.on_notebook_start=<Callable>
    A callable which executes after the kernel manager and kernel client are
    setup, and cells are about to execute. Called with kwargs ``notebook``.
    Default: None
--Execute.record_timing=<Bool>
    If ``True`` (default), then the execution timings of each cell will be
    stored in the metadata of the notebook.
    Default: True
--Execute.shell_timeout_interval=<Int>
    The time to wait (in seconds) for Shell output before retrying. This
    generally doesn't need to be set, but if one needs to check for dead kernels
    at a faster rate this can help.
    Default: 5
--Execute.shutdown_kernel=<Enum>
    If ``graceful`` (default), then the kernel is given time to clean up after
    executing all cells, e.g., to execute its ``atexit`` hooks. If
    ``immediate``, then the kernel is signaled to immediately terminate.
    Choices: any of ['graceful', 'immediate']
    Default: 'graceful'
--Execute.skip_cells_with_tag=<Unicode>
    Name of the cell tag to use to denote a cell that should be skipped.
    Default: 'skip-execution'
--Execute.startup_timeout=<Int>
    The time to wait (in seconds) for the kernel to start. If kernel startup
    takes longer, a RuntimeError is raised.
    Default: 60
--Execute.store_widget_state=<Bool>
    If ``True`` (default), then the state of the Jupyter widgets created at the
    kernel will be stored in the metadata of the notebook.
    Default: True
--Execute.timeout_func=<Any>
    A callable which, when given the cell source as input, returns the time to
    wait (in seconds) for output from cell executions. If a cell execution takes
    longer, a TimeoutError is raised.
    Returning ``None`` or ``-1`` will disable the timeout for the cell. Not
    setting ``timeout_func`` will cause the client to default to using the
    ``timeout`` trait for all cells. The ``timeout_func`` trait overrides
    ``timeout`` if it is not ``None``.
    Default: None

GetGrades(NbGraderPreprocessor) options
---------------------------------------
--GetGrades.display_data_priority=<list-item-1>...
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--GetGrades.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

ClearOutputPreprocessor(Preprocessor) options
---------------------------------------------
--ClearOutputPreprocessor.default_language=<Unicode>
    Deprecated default highlight language as of 5.0, please use language_info
    metadata instead
    Default: 'ipython'
--ClearOutputPreprocessor.display_data_priority=<list-item-1>...
    An ordered list of preferred output type, the first encountered will usually
    be used when converting discarding the others.
    Default: ['text/html', 'application/pdf', 'text/latex', 'image/svg+xml...
--ClearOutputPreprocessor.enabled=<Bool>
    Default: False
--ClearOutputPreprocessor.remove_metadata_fields=<set-item-1>...
    Default: {'collapsed', 'scrolled'}

ClearOutput(NbGraderPreprocessor, ClearOutputPreprocessor) options
------------------------------------------------------------------
--ClearOutput.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearOutput.remove_metadata_fields=<set-item-1>...
    Default: {'collapsed', 'scrolled'}

LimitOutput(NbGraderPreprocessor) options
-----------------------------------------
--LimitOutput.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--LimitOutput.max_lines=<Int>
    maximum number of lines of output (-1 means no limit)
    Default: 1000
--LimitOutput.max_traceback=<Int>
    maximum number of traceback lines (-1 means no limit)
    Default: 100

DeduplicateIds(NbGraderPreprocessor) options
--------------------------------------------
--DeduplicateIds.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

ClearHiddenTests(NbGraderPreprocessor) options
----------------------------------------------
--ClearHiddenTests.begin_test_delimeter=<Unicode>
    The delimiter marking the beginning of hidden tests cases
    Default: 'BEGIN HIDDEN TESTS'
--ClearHiddenTests.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearHiddenTests.end_test_delimeter=<Unicode>
    The delimiter marking the end of hidden tests cases
    Default: 'END HIDDEN TESTS'
--ClearHiddenTests.enforce_metadata=<Bool>
    Whether or not to complain if cells containing hidden test regions are not
    marked as grade cells. WARNING: this will potentially cause things to break
    if you are using the full nbgrader pipeline. ONLY disable this option if you
    are only ever planning to use nbgrader assign.
    Default: True

ClearMarkScheme(NbGraderPreprocessor) options
---------------------------------------------
--ClearMarkScheme.begin_mark_scheme_delimeter=<Unicode>
    The delimiter marking the beginning of hidden tests cases
    Default: 'BEGIN MARK SCHEME'
--ClearMarkScheme.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True
--ClearMarkScheme.end_mark_scheme_delimeter=<Unicode>
    The delimiter marking the end of hidden tests cases
    Default: 'END MARK SCHEME'
--ClearMarkScheme.enforce_metadata=<Bool>
    Whether or not to complain if cells containing marking scheme regions are
    not marked as task cells. WARNING: this will potentially cause things to
    break if you are using the full nbgrader pipeline. ONLY disable this option
    if you are only ever planning to use nbgrader assign.
    Default: True

OverwriteKernelspec(NbGraderPreprocessor) options
-------------------------------------------------
--OverwriteKernelspec.enabled=<Bool>
    Whether to use this preprocessor when running nbgrader
    Default: True

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeCollect(Exchange) options
---------------------------------
--ExchangeCollect.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeCollect.before_duedate=<Bool>
    Collect the last submission before due date or the last submission if no
    submission before due date.
    Default: False
--ExchangeCollect.check_owner=<Bool>
    Whether to cross-check the student_id with the UNIX-owner of the submitted
    directory.
    Default: True
--ExchangeCollect.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeCollect.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
--ExchangeCollect.update=<Bool>
    Update existing submissions with ones that have newer timestamps.
    Default: False

ExchangeFetchAssignment(Exchange) options
-----------------------------------------
--ExchangeFetchAssignment.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetchAssignment.replace_missing_files=<Bool>
    Whether to replace missing files on fetch
    Default: False
--ExchangeFetchAssignment.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetchAssignment.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeFetch(ExchangeFetchAssignment) options
----------------------------------------------
--ExchangeFetch.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetch.replace_missing_files=<Bool>
    Whether to replace missing files on fetch
    Default: False
--ExchangeFetch.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetch.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeFetchFeedback(Exchange) options
---------------------------------------
--ExchangeFetchFeedback.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetchFeedback.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetchFeedback.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeList(Exchange) options
------------------------------
--ExchangeList.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeList.cached=<Bool>
    List assignments in submission cache.
    Default: False
--ExchangeList.inbound=<Bool>
    List inbound files rather than outbound.
    Default: False
--ExchangeList.remove=<Bool>
    Remove, rather than list files.
    Default: False
--ExchangeList.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeList.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeReleaseAssignment(Exchange) options
-------------------------------------------
--ExchangeReleaseAssignment.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeReleaseAssignment.force=<Bool>
    Force overwrite existing files in the exchange.
    Default: False
--ExchangeReleaseAssignment.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeReleaseAssignment.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeRelease(ExchangeReleaseAssignment) options
--------------------------------------------------
--ExchangeRelease.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeRelease.force=<Bool>
    Force overwrite existing files in the exchange.
    Default: False
--ExchangeRelease.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeRelease.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeReleaseFeedback(Exchange) options
-----------------------------------------
--ExchangeReleaseFeedback.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeReleaseFeedback.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeReleaseFeedback.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeSubmit(Exchange) options
--------------------------------
--ExchangeSubmit.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeSubmit.strict=<Bool>
    Whether or not to submit the assignment if there are missing notebooks from
    the released assignment notebooks.
    Default: False
--ExchangeSubmit.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeSubmit.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

BaseConverter(LoggingConfigurable) options
------------------------------------------
--BaseConverter.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--BaseConverter.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--BaseConverter.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--BaseConverter.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--BaseConverter.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None

GenerateAssignment(BaseConverter) options
-----------------------------------------
--GenerateAssignment.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--GenerateAssignment.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateAssignment.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateAssignment.no_database=<Bool>
    Do not save information about the assignment into the database.
    Default: False
--GenerateAssignment.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateAssignment.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateAssignment.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateAssignment.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Assign(GenerateAssignment) options
----------------------------------
--Assign.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--Assign.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Assign.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--Assign.no_database=<Bool>
    Do not save information about the assignment into the database.
    Default: False
--Assign.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--Assign.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Assign.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Assign.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Autograde(BaseConverter) options
--------------------------------
--Autograde.autograde_preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.execute.Execute'>, <class 'nb...
--Autograde.create_student=<Bool>
    Whether to create the student at runtime if it does not already exist.
    Default: True
--Autograde.exclude_overwriting=<key-1>=<value-1>...
    A dictionary with keys corresponding to assignment names and values being a
    list of filenames (relative to the assignment's source directory) that
    should NOT be overwritten with the source version. This is to allow students
    to e.g. edit a python file and submit it alongside the notebooks in their
    assignment.
    Default: {}
--Autograde.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Autograde.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--Autograde.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--Autograde.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Autograde.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Autograde.sanitize_preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.clearoutput.ClearOutput'>, <c...

GenerateFeedback(BaseConverter) options
---------------------------------------
--GenerateFeedback.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateFeedback.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateFeedback.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateFeedback.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateFeedback.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateFeedback.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.getgrades.GetGrades'>, <class...

Feedback(GenerateFeedback) options
----------------------------------
--Feedback.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Feedback.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--Feedback.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--Feedback.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Feedback.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Feedback.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.getgrades.GetGrades'>, <class...

GenerateSolution(BaseConverter) options
---------------------------------------
--GenerateSolution.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--GenerateSolution.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateSolution.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateSolution.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateSolution.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateSolution.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateSolution.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Instructor commands¶

nbgrader quickstart¶

Create an example class files directory with an example config file and
assignment

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--force
    Overwrite existing files if they already exist. WARNING: this is
    equivalent to doing:

        rm -r <course_id>
        nbgrader quickstart <course_id>

    So be careful when using this flag!
    Equivalent to: [--QuickStartApp.force=True]
-f
    Overwrite existing files if they already exist. WARNING: this is
    equivalent to doing:

        rm -r <course_id>
        nbgrader quickstart <course_id>

    So be careful when using this flag!
    Equivalent to: [--QuickStartApp.force=True]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

QuickStartApp(NbGrader) options
-------------------------------
--QuickStartApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--QuickStartApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--QuickStartApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--QuickStartApp.force=<Bool>
    Whether to overwrite existing files
    Default: False
--QuickStartApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--QuickStartApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--QuickStartApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--QuickStartApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--QuickStartApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--QuickStartApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--QuickStartApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

Examples
--------

    You can run `nbgrader quickstart` just on its own from where ever you
            would like to create the example class files directory. It takes just
            one argument, which is the name of your course:

                nbgrader quickstart course101

            Note that this class name need not necessarily be the same as the
            `CourseDirectory.course_id` configuration option, however by default, the
            quickstart command will set `CourseDirectory.course_id` to be the name you give
            on the command line. If you want to use a different folder name, go
            ahead and just provide the name of the folder where your class files
            will be stored, e.g.:

                nbgrader quickstart "World Music"

            and then after running the quickstart commmand, you can edit the
            `nbgrader_config.py` file to change `CourseDirectory.course_id`.

nbgrader generate assignment¶

Produce the version of an assignment to be released to students.

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--no-db
    Do not save information into the database.
    Equivalent to: [--SaveCells.enabled=False --GenerateAssignment.no_database=True]
--no-metadata
    Do not validate or modify cell metatadata.
    Equivalent to: [--ClearSolutions.enforce_metadata=False --ClearHiddenTests.enforce_metadata=False --ClearMarkScheme.enforce_metadata=False --CheckCellMetadata.enabled=False --ComputeChecksums.enabled=False]
--create
    Deprecated: Create an entry for the assignment in the database, if one does not already exist. This is now the default.
    Equivalent to: [--GenerateAssignment.create_assignment=True]
--force
    Overwrite an assignment/submission if it already exists.
    Equivalent to: [--BaseConverter.force=True]
-f
    Overwrite an assignment/submission if it already exists.
    Equivalent to: [--BaseConverter.force=True]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

GenerateAssignmentApp(NbGrader) options
---------------------------------------
--GenerateAssignmentApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateAssignmentApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateAssignmentApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateAssignmentApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateAssignmentApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateAssignmentApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateAssignmentApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateAssignmentApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateAssignmentApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateAssignmentApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

BaseConverter(LoggingConfigurable) options
------------------------------------------
--BaseConverter.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--BaseConverter.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--BaseConverter.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--BaseConverter.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--BaseConverter.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None

GenerateAssignment(BaseConverter) options
-----------------------------------------
--GenerateAssignment.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--GenerateAssignment.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateAssignment.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateAssignment.no_database=<Bool>
    Do not save information about the assignment into the database.
    Default: False
--GenerateAssignment.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateAssignment.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateAssignment.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateAssignment.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Examples
--------

    Produce the version of the assignment that is intended to be released to
            students. This performs several modifications to the original assignment:

                1. It inserts a header and/or footer to each notebook in the
                   assignment, if the header/footer are specified.

                2. It locks certain cells so that they cannot be deleted by students
                   accidentally (or on purpose!)

                3. It removes solutions from the notebooks and replaces them with
                   code or text stubs saying (for example) "YOUR ANSWER HERE".

                4. It clears all outputs from the cells of the notebooks.

                5. It saves information about the cell contents so that we can warn
                   students if they have changed the tests, or if they have failed
                   to provide a response to a written answer. Specifically, this is
                   done by computing a checksum of the cell contents and saving it
                   into the cell metadata.

                6. It saves the tests used to grade students' code into a database,
                   so that those tests can be replaced during autograding if they
                   were modified by the student (you can prevent this by passing the
                   --no-db flag).

                   If the assignment is not already present in the database, it
                   will be automatically created when running `nbgrader generate_assignment`.

            `nbgrader generate_assignment` takes one argument (the name of the assignment), and
            looks for notebooks in the 'source' directory by default, according to
            the directory structure specified in `CourseDirectory.directory_structure`.
            The student version is then saved into the 'release' directory.

            Note that the directory structure requires the `student_id` to be given;
            however, there is no student ID at this point in the process. Instead,
            `nbgrader generate_assignment` sets the student ID to be '.' so by default, files are
            read in according to:

                source/./{assignment_id}/{notebook_id}.ipynb

            and saved according to:

                release/./{assignment_id}/{notebook_id}.ipynb

nbgrader validate¶

Validate a notebook by running it

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--invert
    Complain when cells pass, rather than vice versa.
    Equivalent to: [--Validator.invert=True]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ValidateApp(NbGrader) options
-----------------------------
--ValidateApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ValidateApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ValidateApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ValidateApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ValidateApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ValidateApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ValidateApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ValidateApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ValidateApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ValidateApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Validator(LoggingConfigurable) options
--------------------------------------
--Validator.changed_warning=<Unicode>
    Warning to display when a cell has changed.
    Default: "THE CONTENTS OF {num_changed} TEST CELL(S) HAVE CHANGED!\nTh...
--Validator.failed_warning=<Unicode>
    Warning to display when a cell fails.
    Default: 'VALIDATION FAILED ON {num_failed} CELL(S)! If you submit\nyo...
--Validator.ignore_checksums=<Bool>
    Don't complain if cell checksums have changed (if they are locked cells) or
    haven't changed (if they are solution cells). Note that this will NOT ignore
    changes to cell types.
    Default: False
--Validator.indent=<Unicode>
    A string containing whitespace that will be used to indent code and errors
    Default: '    '
--Validator.invert=<Bool>
    Complain when cells pass, rather than fail.
    Default: False
--Validator.passed_warning=<Unicode>
    Warning to display when a cell passes (when invert=True)
    Default: 'NOTEBOOK PASSED ON {num_passed} CELL(S)!\n'
--Validator.type_changed_warning=<Unicode>
    Warning to display when a cell's type has changed.
    Default: "THE TYPES OF {num_changed} CELL(S) HAVE CHANGED!\nThis might...
--Validator.validate_all=<Bool>
    Validate all cells, not just the graded tests cells.
    Default: False
--Validator.width=<Int>
    Maximum line width for displaying code/errors
    Default: 90

Examples
--------

    You can run `nbgrader validate` on just a single file, e.g.:
                nbgrader validate "Problem 1.ipynb"

            Or, you can run it on multiple files using shell globs:
                nbgrader validate "Problem Set 1/*.ipynb"

            If you want to test instead that none of the tests pass (rather than that
            all of the tests pass, which is the default), you can use --invert:
                nbgrader validate --invert "Problem 1.ipynb"

nbgrader update¶

Update nbgrader notebook metadata

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

UpdateApp(NbGrader) options
---------------------------
--UpdateApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--UpdateApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--UpdateApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--UpdateApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--UpdateApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--UpdateApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--UpdateApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--UpdateApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--UpdateApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--UpdateApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False
--UpdateApp.validate=<Bool>
    whether to validate metadata after updating it
    Default: True

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Examples
--------

    nbgrader stores metadata in the JSON source of the notebooks. Previously,
            we did not do a good job of validating whether this metadata was
            correctly formatted or not. Starting in version 0.4.0 of nbgrader, we
            are explicitly validating this metadata. This will require that you
            update the metadata in your old nbgrader notebooks to be consistent
            with what nbgrader expects.

            The `nbgrader update` command performs this metadata update for you
            easily. All you need to do is point it at a directory, and it will
            find all notebooks in that directory and update them to have the
            correct metadata:

                # update notebooks rooted in the current directory
                nbgrader update .

                # update notebooks rooted in the `class_files` directory
                nbgrader update class_files/

            Alternately, you can open all your notebooks with the "Create Assignment"
            toolbar and re-save them from the notebook interface. But, it will be
            more efficient to run the `nbgrader update` command to get them all in
            one fell swoop.

nbgrader autograde¶

Autograde a notebook by running it

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--create
    Deprecated: Create an entry for the student in the database, if one does not already exist. This is now the default.
    Equivalent to: [--Autograde.create_student=True]
--no-execute
    Don't execute notebooks and clear output when autograding.
    Equivalent to: [--Execute.enabled=False --ClearOutput.enabled=False]
--force
    Overwrite an assignment/submission if it already exists.
    Equivalent to: [--BaseConverter.force=True]
-f
    Overwrite an assignment/submission if it already exists.
    Equivalent to: [--BaseConverter.force=True]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

AutogradeApp(NbGrader) options
------------------------------
--AutogradeApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--AutogradeApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--AutogradeApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--AutogradeApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--AutogradeApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--AutogradeApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--AutogradeApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--AutogradeApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--AutogradeApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--AutogradeApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

BaseConverter(LoggingConfigurable) options
------------------------------------------
--BaseConverter.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--BaseConverter.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--BaseConverter.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--BaseConverter.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--BaseConverter.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None

Autograde(BaseConverter) options
--------------------------------
--Autograde.autograde_preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.execute.Execute'>, <class 'nb...
--Autograde.create_student=<Bool>
    Whether to create the student at runtime if it does not already exist.
    Default: True
--Autograde.exclude_overwriting=<key-1>=<value-1>...
    A dictionary with keys corresponding to assignment names and values being a
    list of filenames (relative to the assignment's source directory) that
    should NOT be overwritten with the source version. This is to allow students
    to e.g. edit a python file and submit it alongside the notebooks in their
    assignment.
    Default: {}
--Autograde.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--Autograde.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--Autograde.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--Autograde.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Autograde.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--Autograde.sanitize_preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.clearoutput.ClearOutput'>, <c...

Examples
--------

    Autograde submitted assignments. This takes one argument for the
            assignment id, and then (by default) autogrades assignments from the
            following directory structure:

                submitted/*/{assignment_id}/*.ipynb

            and saves the autograded files to the corresponding directory in:

                autograded/{student_id}/{assignment_id}/{notebook_id}.ipynb

            The student IDs will be created in the database if they don't already
            exist.

            Note that the assignment must also be present in the database. If it is
            not, you should first create it using `nbgrader generate_assignment`. Then, during
            autograding, the cells that contain tests for the students' answers will
            be overwritten with the master version of the tests that is saved in the
            database (this prevents students from modifying the tests in order to
            improve their score).

            To grade all submissions for "Problem Set 1":

                nbgrader autograde "Problem Set 1"

            To grade only the submission by student with ID 'Hacker':

                nbgrader autograde "Problem Set 1" --student Hacker

            To grade only the notebooks that start with '1':

                nbgrader autograde "Problem Set 1" --notebook "1*"

            By default, student submissions are re-executed and their output cleared.
            For long running notebooks, it can be useful to disable this with the
            '--no-execute' flag:

                nbgrader autograde "Problem Set 1" --no-execute

            Note, however, that doing so will not guarantee that students' solutions
            are correct. If you use this flag, you should make sure you manually
            check all solutions. For example, if a student saved their notebook with
            all outputs cleared, then using --no-execute would result in them
            receiving full credit on all autograded problems.

nbgrader generate feedback¶

Generate feedback from a graded notebook

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--force
    Overwrite an assignment/submission if it already exists.
    Equivalent to: [--BaseConverter.force=True]
-f
    Overwrite an assignment/submission if it already exists.
    Equivalent to: [--BaseConverter.force=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

GenerateFeedbackApp(NbGrader) options
-------------------------------------
--GenerateFeedbackApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateFeedbackApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateFeedbackApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateFeedbackApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateFeedbackApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateFeedbackApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateFeedbackApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateFeedbackApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateFeedbackApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateFeedbackApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

BaseConverter(LoggingConfigurable) options
------------------------------------------
--BaseConverter.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--BaseConverter.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--BaseConverter.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--BaseConverter.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--BaseConverter.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None

GenerateFeedback(BaseConverter) options
---------------------------------------
--GenerateFeedback.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateFeedback.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateFeedback.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateFeedback.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateFeedback.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateFeedback.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.getgrades.GetGrades'>, <class...

Examples
--------

    Create HTML feedback for students after all the grading is finished.
            This takes a single parameter, which is the assignment ID, and then (by
            default) looks at the following directory structure:

                autograded/*/{assignment_id}/*.ipynb

            from which it generates feedback the the corresponding directories
            according to:

                feedback/{student_id}/{assignment_id}/{notebook_id}.html

            Running `nbgrader generate_feedback` requires that `nbgrader autograde` (and most
            likely `nbgrader formgrade`) have been run and that all grading is
            complete.

            To generate feedback for all submissions for "Problem Set 1":
                nbgrader generate_feedback "Problem Set 1"

            To generate feedback only for the student with ID 'Hacker':
                nbgrader generate_feedback "Problem Set 1" --student Hacker

            To feedback for only the notebooks that start with '1':
                nbgrader generate_feedback "Problem Set 1" --notebook "1*"

nbgrader generate solution¶

Produce the solution of an assignment to be released to students.

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--force
    Overwrite the solution if it already exists.
    Equivalent to: [--BaseConverter.force=True]
-f
    Overwrite the solution if it already exists.
    Equivalent to: [--BaseConverter.force=True]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

GenerateSolutionApp(NbGrader) options
-------------------------------------
--GenerateSolutionApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--GenerateSolutionApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--GenerateSolutionApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--GenerateSolutionApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--GenerateSolutionApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--GenerateSolutionApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--GenerateSolutionApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--GenerateSolutionApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--GenerateSolutionApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--GenerateSolutionApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

BaseConverter(LoggingConfigurable) options
------------------------------------------
--BaseConverter.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--BaseConverter.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--BaseConverter.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--BaseConverter.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--BaseConverter.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None

GenerateSolution(BaseConverter) options
---------------------------------------
--GenerateSolution.create_assignment=<Bool>
    Whether to create the assignment at runtime if it does not already exist.
    Default: True
--GenerateSolution.exporter_class=<Type>
    Default: 'nbconvert.exporters.notebook.NotebookExporter'
--GenerateSolution.force=<Bool>
    Whether to overwrite existing assignments/submissions
    Default: False
--GenerateSolution.permissions=<Int>
    Permissions to set on files output by nbgrader. The default is generally
    read-only (444), with the exception of nbgrader generate_assignment and
    nbgrader generate_feedback, in which case the user also has write
    permission.
    Default: 0
--GenerateSolution.post_convert_hook=<Any>
    An optional hook function that you can implement to do some work after
    converting.  This function is called after the notebooks are converted and
    should be used for specific converters such as Autograde, GenerateAssignment
    or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateSolution.pre_convert_hook=<Any>
    An optional hook function that you can implement to do some bootstrapping
    work before converting.  This function is called before the notebooks are
    converted and should be used for specific converters such as Autograde,
    GenerateAssignment or GenerateFeedback.
    It will be called as (all arguments are passed as keywords)::
        hook(assignment=assignment, student=student, notebooks=notebooks)
    Default: None
--GenerateSolution.preprocessors=<list-item-1>...
    Default: [<class 'nbgrader.preprocessors.headerfooter.IncludeHeaderFoo...

Examples
--------

    Produce the solution of the assignment that is intended to be released to
            students. This essentially cleans up the assignment and runs all the cells providing
            the solution conceived by the instructor.

            `nbgrader generate_solution` takes one argument (the name of the assignment), and
            looks for notebooks in the 'source' directory by default, according to
            the directory structure specified in `CourseDirectory.directory_structure`.
            The student version is then saved into the 'solution' directory.

            Note that the directory structure requires the `student_id` to be given;
            however, there is no student ID also at this point in the process. Instead,
            `nbgrader generate_solution` sets the student ID to be '.' so by default, files are
            read in according to:

                source/./{assignment_id}/{notebook_id}.ipynb

            and saved according to:

                solution/./{assignment_id}/{notebook_id}.ipynb

nbgrader export¶

Export information from the database to another format.

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--to=<Unicode>
    destination to export to
    Default: ''
    Equivalent to: [--ExportPlugin.to]
--exporter=<Type>
    The plugin class for exporting the grades.
    Default: 'nbgrader.plugins.export.CsvExportPlugin'
    Equivalent to: [--ExportApp.plugin_class]
--assignment=<list-item-1>...
    list of assignments to export
    Default: []
    Equivalent to: [--ExportPlugin.assignment]
--student=<list-item-1>...
    list of students to export
    Default: []
    Equivalent to: [--ExportPlugin.student]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

ExportApp(NbGrader) options
---------------------------
--ExportApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ExportApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ExportApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ExportApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ExportApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ExportApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ExportApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ExportApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ExportApp.plugin_class=<Type>
    The plugin class for exporting the grades.
    Default: 'nbgrader.plugins.export.CsvExportPlugin'
--ExportApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ExportApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExportPlugin(BasePlugin) options
--------------------------------
--ExportPlugin.assignment=<list-item-1>...
    list of assignments to export
    Default: []
--ExportPlugin.student=<list-item-1>...
    list of students to export
    Default: []
--ExportPlugin.to=<Unicode>
    destination to export to
    Default: ''

Examples
--------

    The default is to export to a file called "grades.csv", i.e.:

                nbgrader export

            You can customize the filename with the --to flag:

                nbgrader export --to mygrades.csv

            You can export the grades for a single (or limited set) of students
            or assignments with the --assignment and/or --student flag:

                nbgrader export --assignment [assignmentID]
                                --student [studentID1,studentID2]

            Where the studentIDs and assignmentIDs are a list of IDs and
            assignments. The assignments or studentIDs need to quoted if they
            contain not only numbers. The square brackets are obligatory.

            To change the export type, you will need a class that inherits from
            nbgrader.plugins.ExportPlugin. If your exporter is named
            `MyCustomExporter` and is saved in the file `myexporter.py`, then:

                nbgrader export --exporter=myexporter.MyCustomExporter

The following commands are used to manage the database:

nbgrader db student add¶

Add a student to the nbgrader database

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--last-name=<Unicode>
    The last name of the student
    Default: None
    Equivalent to: [--DbStudentAddApp.last_name]
--first-name=<Unicode>
    The first name of the student
    Default: None
    Equivalent to: [--DbStudentAddApp.first_name]
--email=<Unicode>
    The email of the student
    Default: None
    Equivalent to: [--DbStudentAddApp.email]
--lms-user-id=<Unicode>
    The LMS user id of the student
    Default: None
    Equivalent to: [--DbStudentAddApp.lms_user_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbStudentAddApp(DbBaseApp) options
----------------------------------
--DbStudentAddApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbStudentAddApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbStudentAddApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbStudentAddApp.email=<Unicode>
    The email of the student
    Default: None
--DbStudentAddApp.first_name=<Unicode>
    The first name of the student
    Default: None
--DbStudentAddApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbStudentAddApp.last_name=<Unicode>
    The last name of the student
    Default: None
--DbStudentAddApp.lms_user_id=<Unicode>
    The LMS user id of the student
    Default: None
--DbStudentAddApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbStudentAddApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbStudentAddApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbStudentAddApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbStudentAddApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbStudentAddApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

nbgrader db student import¶

Import students into the nbgrader database from a CSV file

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbGenericImportApp(DbBaseApp) options
-------------------------------------
--DbGenericImportApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbGenericImportApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbGenericImportApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbGenericImportApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbGenericImportApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbGenericImportApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbGenericImportApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbGenericImportApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbGenericImportApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbGenericImportApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbStudentImportApp(DbGenericImportApp) options
----------------------------------------------
--DbStudentImportApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbStudentImportApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbStudentImportApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbStudentImportApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbStudentImportApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbStudentImportApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbStudentImportApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbStudentImportApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbStudentImportApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbStudentImportApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Examples
--------

    This command imports a CSV file into the database. The columns of
    the CSV file must match the names of the columns in the database.
    All columns are optional, except the columns corresponding to the
    unique identifier of the Student. The keys/column names that are
    expected are the following:

      - id (required)
      - first_name
      - last_name
      - email
      - lms_user_id

nbgrader db student remove¶

Remove a student from the nbgrader database

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--force
    Complete the operation, even if it means grades will be deleted.
    Equivalent to: [--DbStudentRemoveApp.force=True]
-f
    Complete the operation, even if it means grades will be deleted.
    Equivalent to: [--DbStudentRemoveApp.force=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbStudentRemoveApp(DbBaseApp) options
-------------------------------------
--DbStudentRemoveApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbStudentRemoveApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbStudentRemoveApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbStudentRemoveApp.force=<Bool>
    Confirm operation if it means grades will be deleted.
    Default: False
--DbStudentRemoveApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbStudentRemoveApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbStudentRemoveApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbStudentRemoveApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbStudentRemoveApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbStudentRemoveApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbStudentRemoveApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

nbgrader db student list¶

List students in the nbgrader database

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbStudentListApp(DbBaseApp) options
-----------------------------------
--DbStudentListApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbStudentListApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbStudentListApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbStudentListApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbStudentListApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbStudentListApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbStudentListApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbStudentListApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbStudentListApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbStudentListApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

nbgrader db assignment add¶

Add an assignment to the nbgrader database

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--duedate=<Unicode>
    The due date of the assignment
    Default: None
    Equivalent to: [--DbAssignmentAddApp.duedate]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbAssignmentAddApp(DbBaseApp) options
-------------------------------------
--DbAssignmentAddApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbAssignmentAddApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbAssignmentAddApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbAssignmentAddApp.duedate=<Unicode>
    The due date of the assignment
    Default: None
--DbAssignmentAddApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbAssignmentAddApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbAssignmentAddApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbAssignmentAddApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbAssignmentAddApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbAssignmentAddApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbAssignmentAddApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

nbgrader db assignment import¶

Import assignments into the nbgrader database from a CSV file

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbGenericImportApp(DbBaseApp) options
-------------------------------------
--DbGenericImportApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbGenericImportApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbGenericImportApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbGenericImportApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbGenericImportApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbGenericImportApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbGenericImportApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbGenericImportApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbGenericImportApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbGenericImportApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbAssignmentImportApp(DbGenericImportApp) options
-------------------------------------------------
--DbAssignmentImportApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbAssignmentImportApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbAssignmentImportApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbAssignmentImportApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbAssignmentImportApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbAssignmentImportApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbAssignmentImportApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbAssignmentImportApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbAssignmentImportApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbAssignmentImportApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Examples
--------

    This command imports a CSV file into the database. The columns of
    the CSV file must match the names of the columns in the database.
    All columns are optional, except the columns corresponding to the
    unique identifier of the Assignment. The keys/column names that are
    expected are the following:

      - name (required)
      - duedate
      - course_id

nbgrader db assignment remove¶

Remove an assignment from the nbgrader database

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--force
    Complete the operation, even if it means grades will be deleted.
    Equivalent to: [--DbAssignmentRemoveApp.force=True]
-f
    Complete the operation, even if it means grades will be deleted.
    Equivalent to: [--DbAssignmentRemoveApp.force=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbAssignmentRemoveApp(DbBaseApp) options
----------------------------------------
--DbAssignmentRemoveApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbAssignmentRemoveApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbAssignmentRemoveApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbAssignmentRemoveApp.force=<Bool>
    Confirm operation if it means grades will be deleted.
    Default: False
--DbAssignmentRemoveApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbAssignmentRemoveApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbAssignmentRemoveApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbAssignmentRemoveApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbAssignmentRemoveApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbAssignmentRemoveApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbAssignmentRemoveApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

nbgrader db assignment list¶

List assignments in the nbgrader database

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbBaseApp(NbGrader) options
---------------------------
--DbBaseApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbBaseApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbBaseApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbBaseApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbBaseApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbBaseApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbBaseApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbBaseApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbBaseApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbBaseApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

DbAssignmentListApp(DbBaseApp) options
--------------------------------------
--DbAssignmentListApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--DbAssignmentListApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--DbAssignmentListApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--DbAssignmentListApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--DbAssignmentListApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--DbAssignmentListApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--DbAssignmentListApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--DbAssignmentListApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--DbAssignmentListApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--DbAssignmentListApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

The following commands are meant for instructors, but are only relevant when using nbgrader in a shared server environment:

nbgrader release assignment¶

Release an assignment to the nbgrader exchange

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--force
    Force overwrite of existing files in the exchange.
    Equivalent to: [--ExchangeReleaseAssignment.force=True]
-f
    Force overwrite of existing files in the exchange.
    Equivalent to: [--ExchangeReleaseAssignment.force=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
    Equivalent to: [--Exchange.timezone]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ReleaseAssignmentApp(NbGrader) options
--------------------------------------
--ReleaseAssignmentApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ReleaseAssignmentApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ReleaseAssignmentApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ReleaseAssignmentApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ReleaseAssignmentApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ReleaseAssignmentApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ReleaseAssignmentApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ReleaseAssignmentApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ReleaseAssignmentApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ReleaseAssignmentApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeReleaseAssignment(Exchange) options
-------------------------------------------
--ExchangeReleaseAssignment.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeReleaseAssignment.force=<Bool>
    Force overwrite existing files in the exchange.
    Default: False
--ExchangeReleaseAssignment.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeReleaseAssignment.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

Examples
--------

    Release an assignment to students. For the usage of instructors.

            This command is run from the top-level nbgrader folder. Before running
            this command, there are two things you must do.

            First, you have to set the unique `course_id` for the course. It must be
            unique for each instructor/course combination. To set it in the config
            file add a line to the `nbgrader_config.py` file:

                c.CourseDirectory.course_id = 'phys101'

            To pass the `course_id` at the command line, add `--course=phys101` to any
            of the below commands.

            Second, the assignment to be released must already be in the `release` folder.
            The usual way of getting an assignment into this folder is by running
            `nbgrader generate_assignment`.

            To release an assignment named `assignment1` run:

                nbgrader release_assignment assignment1

            If the assignment has already been released, you will have to add the
            `--force` flag to overwrite the released assignment:

                nbgrader release_assignment --force assignment1

            To query the exchange to see a list of your released assignments:

                nbgrader list

nbgrader release feedback¶

Release assignment feedback to the nbgrader exchange

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
    Equivalent to: [--Exchange.timezone]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ReleaseFeedbackApp(NbGrader) options
------------------------------------
--ReleaseFeedbackApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ReleaseFeedbackApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ReleaseFeedbackApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ReleaseFeedbackApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ReleaseFeedbackApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ReleaseFeedbackApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ReleaseFeedbackApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ReleaseFeedbackApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ReleaseFeedbackApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ReleaseFeedbackApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeReleaseFeedback(Exchange) options
-----------------------------------------
--ExchangeReleaseFeedback.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeReleaseFeedback.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeReleaseFeedback.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

Examples
--------

    Release feedback for an assignment to students. For the usage of instructors.

            This command is run from the top-level nbgrader folder.

            The command releases the feedback present in the `feedback` folder. To populate
            this folder use the `nbgrader generate_feedback` command.

            To release the feedback for an assignment named `assignment1` run:

                nbgrader release_feedback assignment1

            Release feedback overrides existing files. It should not be a problem given
            that the feedback is associated with the hash of hte notebook. Any new notebook
            will map to a different file. The only way a file actually gets replaced is when
            the same input notebook gets re-graded and in these cases one would want the latest
            grading to be the right one.

nbgrader list¶

List assignments in the nbgrader exchange

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--inbound
    List inbound files rather than outbound.
    Equivalent to: [--ExchangeList.inbound=True]
--cached
    List cached files rather than inbound/outbound.
    Equivalent to: [--ExchangeList.cached=True]
--remove
    Remove an assignment from the exchange.
    Equivalent to: [--ExchangeList.remove=True]
--json
    Print out assignments as json.
    Equivalent to: [--ExchangeList.as_json=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
    Equivalent to: [--Exchange.timezone]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ListApp(NbGrader) options
-------------------------
--ListApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ListApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ListApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ListApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ListApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ListApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ListApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ListApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ListApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ListApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeList(Exchange) options
------------------------------
--ExchangeList.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeList.cached=<Bool>
    List assignments in submission cache.
    Default: False
--ExchangeList.inbound=<Bool>
    List inbound files rather than outbound.
    Default: False
--ExchangeList.remove=<Bool>
    Remove, rather than list files.
    Default: False
--ExchangeList.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeList.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

Examples
--------

    List assignments in the nbgrader exchange. For the usage of instructors
            and students.

            Students
            ========

            To list assignments for a course, you must first know the `course_id` for
            your course. If you don't know it, ask your instructor.

            To list the released assignments for the course `phys101`:

                nbgrader list phys101

            Instructors
            ===========

            To list outbound (released) or inbound (submitted) assignments for a course,
            you must configure the `course_id` in your config file or the command line.

            To see all of the released assignments, run

                nbgrader list  # course_id in the config file

            or

                nbgrader list --course phys101  # course_id provided

            To see the inbound (submitted) assignments:

                nbgrader list --inbound

            You can use the `--student` and `--assignment` options to filter the list
            by student or assignment:

                nbgrader list --inbound --student=student1 --assignment=assignment1

            If a student has submitted an assignment multiple times, the `list` command
            will show all submissions with their timestamps.

            The `list` command can optionally remove listed assignments by providing the
            `--remove` flag:

                nbgrader list --inbound --remove --student=student1

nbgrader collect¶

Collect an assignment from the nbgrader exchange

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--update
    Update existing submissions with ones that have newer timestamps.
    Equivalent to: [--ExchangeCollect.update=True]
--before-duedate
    Collect the last submission before due date or the last submission if no submission before due date.
    Equivalent to: [--ExchangeCollect.before_duedate=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
    Equivalent to: [--Exchange.timezone]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

CollectApp(NbGrader) options
----------------------------
--CollectApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--CollectApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--CollectApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--CollectApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--CollectApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--CollectApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--CollectApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--CollectApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--CollectApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--CollectApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeCollect(Exchange) options
---------------------------------
--ExchangeCollect.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeCollect.before_duedate=<Bool>
    Collect the last submission before due date or the last submission if no
    submission before due date.
    Default: False
--ExchangeCollect.check_owner=<Bool>
    Whether to cross-check the student_id with the UNIX-owner of the submitted
    directory.
    Default: True
--ExchangeCollect.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeCollect.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
--ExchangeCollect.update=<Bool>
    Update existing submissions with ones that have newer timestamps.
    Default: False

Examples
--------

    Collect assignments students have submitted. For the usage of instructors.

            This command is run from the top-level nbgrader folder. Before running
            this command, you may want toset the unique `course_id` for the course.
            It must be unique for each instructor/course combination. To set it in
            the config file add a line to the `nbgrader_config.py` file:

                c.CourseDirectory.course_id = 'phys101'

            To pass the `course_id` at the command line, add `--course=phys101` to any
            of the below commands.

            To collect `assignment1` for all students:

                nbgrader collect assignment1

            To collect `assignment1` for only `student1`:

                nbgrader collect --student=student1 assignment1

            Collected assignments will go into the `submitted` folder with the proper
            directory structure to start grading. All submissions are timestamped and
            students can turn an assignment in multiple times. The `collect` command
            will always get the most recent submission from each student, but it will
            never overwrite an existing submission unless you provide the `--update`
            flag:

                nbgrader collect --update assignment1

nbgrader zip collect¶

Collect assignments from archives (zip files).

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--force
    Force overwrite of existing files.
    Equivalent to: [--ZipCollectApp.force=True --ExtractorPlugin.force=True]
-f
    Force overwrite of existing files.
    Equivalent to: [--ZipCollectApp.force=True --ExtractorPlugin.force=True]
--strict
    Skip submitted notebooks with invalid names.
    Equivalent to: [--ZipCollectApp.strict=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--extractor=<Type>
    The plugin class for extracting the archive files in the
    `archive_directory`.
    Default: 'nbgrader.plugins.zipcollect.ExtractorPlugin'
    Equivalent to: [--ZipCollectApp.extractor_plugin]
--collector=<Type>
    The plugin class for processing the submitted file names after they have
    been extracted into the `extracted_directory`.
    Default: 'nbgrader.plugins.zipcollect.FileNameCollectorPlugin'
    Equivalent to: [--ZipCollectApp.collector_plugin]
--zip_ext=<list-item-1>...
    List of valid archive (zip) filename extensions to extract. Any archive
    (zip) files with an extension not in this list are copied to the
    `extracted_directory`.
    Default: ['.zip', '.gz']
    Equivalent to: [--ExtractorPlugin.zip_ext]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

ExtractorPlugin(BasePlugin) options
-----------------------------------
--ExtractorPlugin.force=<Bool>
    Force overwrite of existing files.
    Default: False
--ExtractorPlugin.zip_ext=<list-item-1>...
    List of valid archive (zip) filename extensions to extract. Any archive
    (zip) files with an extension not in this list are copied to the
    `extracted_directory`.
    Default: ['.zip', '.gz']

FileNameCollectorPlugin(BasePlugin) options
-------------------------------------------
--FileNameCollectorPlugin.named_regexp=<Unicode>
    This regular expression is applied to each submission filename and MUST be
    supplied by the instructor. This regular expression MUST provide the
    `(?P<student_id>...)` and `(?P<file_id>...)` named group expressions.
    Optionally this regular expression can also provide the
    `(?P<first_name>...)`, `(?P<last_name>...)`, `(?P<email>...)`, and
    `(?P<timestamp>...)` named group expressions. For example if the filename
    is:
        `ps1_bitdiddle_attempt_2016-01-30-15-00-00_problem1.ipynb`
    then this `named_regexp` could be:
    ".*_(?P<student_id>\w+)_attempt_(?P<timestamp>[0-9\-]+)_(?P<file_id>\w+)"
    For named group regular expression examples see
    https://docs.python.org/3/howto/regex.html
    Default: ''
--FileNameCollectorPlugin.valid_ext=<list-item-1>...
    List of valid submission filename extensions to collect. Any submitted file
    with an extension not in this list is skipped.
    Default: ['.ipynb']

ZipCollectApp(NbGrader) options
-------------------------------
--ZipCollectApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--ZipCollectApp.archive_directory=<Unicode>
    The name of the directory that contains assignment submission files and/or
    archives (zip) files manually downloaded from a LMS. This corresponds to the
    `collect_step` variable in the `collect_structure` config option.
    Default: 'archive'
--ZipCollectApp.collect_directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the zip collect process. This MUST contain named keys for 'downloaded',
    'assignment_id', and 'collect_step'.
    Default: '{downloaded}/{assignment_id}/{collect_step}'
--ZipCollectApp.collector_plugin=<Type>
    The plugin class for processing the submitted file names after they have
    been extracted into the `extracted_directory`.
    Default: 'nbgrader.plugins.zipcollect.FileNameCollectorPlugin'
--ZipCollectApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--ZipCollectApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--ZipCollectApp.downloaded_directory=<Unicode>
    The main directory that corresponds to the `downloaded` variable in the
    `collect_structure` config option.
    Default: 'downloaded'
--ZipCollectApp.extracted_directory=<Unicode>
    The name of the directory that contains assignment submission files
    extracted or copied from the `archive_directory`. This corresponds to the
    `collect_step` variable in the `collect_structure` config option.
    Default: 'extracted'
--ZipCollectApp.extractor_plugin=<Type>
    The plugin class for extracting the archive files in the
    `archive_directory`.
    Default: 'nbgrader.plugins.zipcollect.ExtractorPlugin'
--ZipCollectApp.force=<Bool>
    Force overwrite of existing files.
    Default: False
--ZipCollectApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--ZipCollectApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--ZipCollectApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--ZipCollectApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--ZipCollectApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--ZipCollectApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--ZipCollectApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False
--ZipCollectApp.strict=<Bool>
    Skip submitted notebooks with invalid names.
    Default: False

Examples
--------

    Collect assignment submissions from files and/or archives (zip) files
            manually downloaded from a LMS. For the usage of instructors.

            This command is run from the top-level nbgrader folder. In order to
            facilitate the collect process, nbgrader places some constraints on how
            the manually downloaded archive (zip) files must be structured. By
            default, the directory structure must look like this:

                {downloaded}/{assignment_id}/{collect_step}

            where `downloaded` is the main directory, `assignment_id` is the name
            of the assignment and `collect_step` is the step in the collect
            process.

            Manually downloaded assignment submissions files and/or archives (zip)
            files must be placed in the `archive_directory`:

                {downloaded}/{assignment_id}/{archive_directory}

            It is expected that the instructor has already created this directory
            and placed the downloaded assignment submissions files and/or archives
            (zip) files in this directory.

            Archive (zip) files in the `archive_directory` will be extracted, and
            any non-archive files will be copied, to the `extracted_directory`:

                {downloaded}/{assignment_id}/{extracted_directory}

            After which the files in the `extracted_directory` will be collected
            and copied into the students `submitted_directory`:

                {submitted_directory}/{student_id}/{assignment_id}/{notebook_id}.ipynb

            By default the collection of files in the `extracted_directory` is
            managed via the :class:`~nbgrader.plugins.zipcollect.FileNameCollectorPlugin`
            plugin. Each filename is sent to the plugin, which in turn returns an
            object containing the `student_id`, `file_id`, `first_name`,
            `last_name`, `email`, and `timestamp` data. For more information run:

                nbgrader zip_collect --help-all

            To change the default plugin, you will need a class that inherits from
            :class:`~nbgrader.plugins.zipcollect.FileNameCollectorPlugin`. If your
            collector is named `MyCustomCollector` and is saved in the file
            `mycollector.py`, then:

                nbgrader zip_collect --collector=mycollector.MyCustomCollector

Student commands¶

Note that these commands are meant for students, but are only relevant when using nbgrader in a shared server environment.

nbgrader fetch assignment¶

Fetch an assignment from the nbgrader exchange

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--replace
    replace missing files, even if the assignment has already been fetched
    Equivalent to: [--ExchangeFetchAssignment.replace_missing_files=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
    Equivalent to: [--Exchange.timezone]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

FetchAssignmentApp(NbGrader) options
------------------------------------
--FetchAssignmentApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--FetchAssignmentApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--FetchAssignmentApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--FetchAssignmentApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--FetchAssignmentApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--FetchAssignmentApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--FetchAssignmentApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--FetchAssignmentApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--FetchAssignmentApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--FetchAssignmentApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeFetchAssignment(Exchange) options
-----------------------------------------
--ExchangeFetchAssignment.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetchAssignment.replace_missing_files=<Bool>
    Whether to replace missing files on fetch
    Default: False
--ExchangeFetchAssignment.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetchAssignment.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

Examples
--------

    Fetch an assignment that an instructor has released. For the usage of students.

            You can run this command from any directory, but usually, you will have a
            directory where you are keeping your course assignments.

            To fetch an assignment by name into the current directory:

                nbgrader fetch_assignment assignment1

            To fetch an assignment for a specific course, you must first know the
            `course_id` for your course.  If you don't know it, ask your instructor.
            Then, simply include the argument with the '--course' flag.

                nbgrader fetch_assignment assignment1 --course=phys101

            This will create an new directory named `assignment1` where you can work
            on the assignment. When you are done, use the `nbgrader submit` command
            to turn in the assignment.

nbgrader fetch feedback¶

Fetch feedback for an assignment from the nbgrader exchange

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
    Equivalent to: [--Exchange.timezone]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

FetchFeedbackApp(NbGrader) options
----------------------------------
--FetchFeedbackApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--FetchFeedbackApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--FetchFeedbackApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--FetchFeedbackApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--FetchFeedbackApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--FetchFeedbackApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--FetchFeedbackApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--FetchFeedbackApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--FetchFeedbackApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--FetchFeedbackApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeFetchFeedback(Exchange) options
---------------------------------------
--ExchangeFetchFeedback.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeFetchFeedback.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeFetchFeedback.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

Examples
--------

    Fetch feedback for an assignment, if an instructor has released it.
            For the usage of students.

            You can run this command from any directory, but usually, you will run
            it from the directory where you are keeping your course assignments.

            To fetch an assignment by name into the current directory:

                nbgrader fetch_feedback assignment1

            To fetch the assignment for a specific course (or if your course_id is
            not set in a configuration file already), you must first know the
            `course_id` for your course.  If you don't know it, ask your
            instructor.  Then, simply include the argument with the '--course'
            flag:

                nbgrader fetch_feedback assignment1 --course=phys101

            This will create a new `feedback` directory within the corresponding
            assignment directory with the feedback inside.  There can be multiple
            feedbacks if you submitted multiple times and the instructor generated
            it multiple times.

nbgrader submit¶

Submit an assignment to the nbgrader exchange

Options
=======
The options below are convenience aliases to configurable class-options,
as listed in the "Equivalent to" description-line of the aliases.
To see all configurable class-options for some <cmd>, use:
    <cmd> --help-all

--debug
    set log level to DEBUG (maximize logging output)
    Equivalent to: [--Application.log_level=DEBUG]
--quiet
    set log level to CRITICAL (minimize logging output)
    Equivalent to: [--Application.log_level=CRITICAL]
--strict
    Fail if the submission is missing notebooks for the assignment
    Equivalent to: [--ExchangeSubmit.strict=True]
--log-level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
    Equivalent to: [--Application.log_level]
--student=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
    Equivalent to: [--CourseDirectory.student_id]
--assignment=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
    Equivalent to: [--CourseDirectory.assignment_id]
--notebook=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
    Equivalent to: [--CourseDirectory.notebook_id]
--db=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
    Equivalent to: [--CourseDirectory.db_url]
--course-dir=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
    Equivalent to: [--CourseDirectory.root]
--timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'
    Equivalent to: [--Exchange.timezone]
--course=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
    Equivalent to: [--CourseDirectory.course_id]

Class options
=============
The command-line option below sets the respective configurable class-parameter:
    --Class.parameter=value
This line is evaluated in Python, so simple expressions are allowed.
For instance, to set `C.a=[0,1,2]`, you may type this:
    --C.a='range(3)'

Application(SingletonConfigurable) options
------------------------------------------
--Application.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--Application.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--Application.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--Application.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--Application.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

JupyterApp(Application) options
-------------------------------
--JupyterApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--JupyterApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--JupyterApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--JupyterApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--JupyterApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--JupyterApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--JupyterApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--JupyterApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--JupyterApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

NbGrader(JupyterApp) options
----------------------------
--NbGrader.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--NbGrader.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--NbGrader.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--NbGrader.generate_config=<Bool>
    Generate default config file.
    Default: False
--NbGrader.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--NbGrader.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--NbGrader.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--NbGrader.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--NbGrader.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--NbGrader.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

SubmitApp(NbGrader) options
---------------------------
--SubmitApp.answer_yes=<Bool>
    Answer yes to any prompts.
    Default: False
--SubmitApp.config_file=<Unicode>
    Full path of a config file.
    Default: ''
--SubmitApp.config_file_name=<Unicode>
    Specify a config file to load.
    Default: ''
--SubmitApp.generate_config=<Bool>
    Generate default config file.
    Default: False
--SubmitApp.log_datefmt=<Unicode>
    The date format used by logging formatters for %(asctime)s
    Default: '%Y-%m-%d %H:%M:%S'
--SubmitApp.log_format=<Unicode>
    The Logging format template
    Default: '[%(name)s]%(highlevel)s %(message)s'
--SubmitApp.log_level=<Enum>
    Set the log level by value or name.
    Choices: any of [0, 10, 20, 30, 40, 50, 'DEBUG', 'INFO', 'WARN', 'ERROR', 'CRITICAL']
    Default: 30
--SubmitApp.logfile=<Unicode>
    Name of the logfile to log to. By default, log output is not written to any
    file.
    Default: ''
--SubmitApp.show_config=<Bool>
    Instead of starting the Application, dump configuration to stdout
    Default: False
--SubmitApp.show_config_json=<Bool>
    Instead of starting the Application, dump configuration to stdout (as JSON)
    Default: False

ExchangeFactory(LoggingConfigurable) options
--------------------------------------------
--ExchangeFactory.collect=<Type>
    A plugin for collecting assignments.
    Default: 'nbgrader.exchange.default.collect.ExchangeCollect'
--ExchangeFactory.exchange=<Type>
    A plugin for exchange.
    Default: 'nbgrader.exchange.default.exchange.Exchange'
--ExchangeFactory.fetch_assignment=<Type>
    A plugin for fetching assignments.
    Default: 'nbgrader.exchange.default.fetch_assignment.ExchangeFetchAssi...
--ExchangeFactory.fetch_feedback=<Type>
    A plugin for fetching feedback.
    Default: 'nbgrader.exchange.default.fetch_feedback.ExchangeFetchFeedback'
--ExchangeFactory.list=<Type>
    A plugin for listing exchange files.
    Default: 'nbgrader.exchange.default.list.ExchangeList'
--ExchangeFactory.release_assignment=<Type>
    A plugin for releasing assignments.
    Default: 'nbgrader.exchange.default.release_assignment.ExchangeRelease...
--ExchangeFactory.release_feedback=<Type>
    A plugin for releasing feedback.
    Default: 'nbgrader.exchange.default.release_feedback.ExchangeReleaseFe...
--ExchangeFactory.submit=<Type>
    A plugin for submitting assignments.
    Default: 'nbgrader.exchange.default.submit.ExchangeSubmit'

CourseDirectory(LoggingConfigurable) options
--------------------------------------------
--CourseDirectory.assignment_id=<Unicode>
    The assignment name. This MUST be specified, either by setting the config
    option, passing an argument on the command line, or using the --assignment
    option on the command line.
    Default: ''
--CourseDirectory.autograded_directory=<Unicode>
    The name of the directory that contains assignment submissions after they
    have been autograded. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'autograded'
--CourseDirectory.course_id=<Unicode>
    A key that is unique per instructor and course. This can be specified,
    either by setting the config option, or using the --course option on the
    command line.
    Default: ''
--CourseDirectory.db_url=<Unicode>
    URL to the database. Defaults to sqlite:///<root>/gradebook.db, where <root>
    is another configurable variable.
    Default: ''
--CourseDirectory.directory_structure=<Unicode>
    Format string for the directory structure that nbgrader works over during
    the grading process. This MUST contain named keys for 'nbgrader_step',
    'student_id', and 'assignment_id'. It SHOULD NOT contain a key for
    'notebook_id', as this will be automatically joined with the rest of the
    path.
    Default: '{nbgrader_step}/{student_id}/{assignment_id}'
--CourseDirectory.feedback_directory=<Unicode>
    The name of the directory that contains assignment feedback after grading
    has been completed. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'feedback'
--CourseDirectory.groupshared=<Bool>
    Make all instructor files group writeable (g+ws, default g+r only) and
    exchange directories group readable/writeable (g+rws, default g=nothing only
    ) by default.  This should only be used if you carefully set the primary
    groups of your notebook servers and fully understand the unix permission
    model.  This changes the default permissions from 444 (unwriteable) to 664
    (writeable), so that other instructors are able to delete/overwrite files.
    Default: False
--CourseDirectory.ignore=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively,
    matching files and directories will be ignored with a debug message.
    Default: ['.ipynb_checkpoints', '*.pyc', '__pycache__', 'feedback']
--CourseDirectory.include=<list-item-1>...
    List of file names or file globs. Upon copying directories recursively, non
    matching files will be ignored with a debug message.
    Default: ['*']
--CourseDirectory.max_file_size=<Int>
    Maximum size of files (in kilobytes; default: 100Mb). Upon copying
    directories recursively, larger files will be ignored with a warning.
    Default: 100000
--CourseDirectory.notebook_id=<Unicode>
    File glob to match notebook names, excluding the '.ipynb' extension. This
    can be changed to filter by notebook.
    Default: '*'
--CourseDirectory.release_directory=<Unicode>
    The name of the directory that contains the version of the assignment that
    will be released to students. This corresponds to the `nbgrader_step`
    variable in the `directory_structure` config option.
    Default: 'release'
--CourseDirectory.root=<Unicode>
    The root directory for the course files (that includes the `source`,
    `release`, `submitted`, `autograded`, etc. directories). Defaults to the
    current working directory.
    Default: ''
--CourseDirectory.solution_directory=<Unicode>
    The name of the directory that contains the assignment solution after
    grading has been completed. This corresponds to the `nbgrader_step` variable
    in the `directory_structure` config option.
    Default: 'solution'
--CourseDirectory.source_directory=<Unicode>
    The name of the directory that contains the master/instructor version of
    assignments. This corresponds to the `nbgrader_step` variable in the
    `directory_structure` config option.
    Default: 'source'
--CourseDirectory.student_id=<Unicode>
    File glob to match student IDs. This can be changed to filter by student.
    Note: this is always changed to '.' when running `nbgrader assign`, as the
    assign step doesn't have any student ID associated with it. With `nbgrader
    submit`, this instead forces the use of an alternative student ID for the
    submission. See `nbgrader submit --help`.
    If the ID is purely numeric and you are passing it as a flag on the command
    line, you will need to escape the quotes in order to have it detected as a
    string, for example `--student=""12345""`. See:
        https://github.com/jupyter/nbgrader/issues/743
    for more details.
    Default: '*'
--CourseDirectory.student_id_exclude=<Unicode>
    Comma-separated list of student IDs to exclude.  Counterpart of student_id.
    This is useful when running commands on all students, but certain students
    cause errors or otherwise must be left out.  Works at least for autograde,
    generate_feedback, and release_feedback.
    Default: ''
--CourseDirectory.submitted_directory=<Unicode>
    The name of the directory that contains assignments that have been submitted
    by students for grading. This corresponds to the `nbgrader_step` variable in
    the `directory_structure` config option.
    Default: 'submitted'

Exchange(LoggingConfigurable) options
-------------------------------------
--Exchange.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--Exchange.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--Exchange.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

ExchangeSubmit(Exchange) options
--------------------------------
--ExchangeSubmit.assignment_dir=<Unicode>
    Local path for storing student assignments.  Defaults to '.' which is
    normally Jupyter's notebook_dir.
    Default: '.'
--ExchangeSubmit.strict=<Bool>
    Whether or not to submit the assignment if there are missing notebooks from
    the released assignment notebooks.
    Default: False
--ExchangeSubmit.timestamp_format=<Unicode>
    Format string for timestamps
    Default: '%Y-%m-%d %H:%M:%S.%f %Z'
--ExchangeSubmit.timezone=<Unicode>
    Timezone for recording timestamps
    Default: 'UTC'

Examples
--------

    Submit an assignment for grading. For the usage of students.

            You must run this command from the directory containing the assignments
            sub-directory. For example, if you want to submit an assignment named
            `assignment1`, that must be a sub-directory of your current working directory.
            If you are inside the `assignment1` directory, it won't work.

            To fetch an assignment you must first know the `course_id` for your course.
            If you don't know it, ask your instructor.

            To submit `assignment1` to the course `phys101`:

                nbgrader submit assignment1 --course phys101

            You can submit an assignment multiple times and the instructor will always
            get the most recent version. Your assignment submission are timestamped
            so instructors can tell when you turned it in. No other students will
            be able to see your submissions.

            Certain setups may require to specify an alternative student id,
            for example to include additional information such as the
            student's group:

                nbgrader submit assignment1 --student a4.jane.doo

            The provided id should contain no `+` signs nor wildcards `*`.
            This feature is typically meant for scripted use. Do *not* use
            it to try to cheat and fake a submission from another student:
            your actual student id can be recovered from the file ownership,
            leaving behind a big trace leading to you.

Using nbgrader with JupyterHub¶

See also

Creating and grading assignments: Documentation for nbgrader generate_assignment, nbgrader autograde, nbgrader formgrade, and nbgrader generate_feedback.
Exchanging assignment files: Documentation for nbgrader release_assignment, nbgrader fetch_assignment, nbgrader submit, and nbgrader collect.
The nbgrader_config.py file: Details on how to setup the nbgrader_config.py file.
The philosophy and the approach: More details on how the nbgrader hierarchy is structured.
JupyterHub Documentation: Detailed documentation describing how JupyterHub works, which is very much required reading if you want to integrate the formgrader with JupyterHub.

For instructors running a class with JupyterHub, nbgrader offers several tools that optimize and enrich the instructors’ and students’ experience of sharing the same system. By integrating with JupyterHub, nbgrader streamlines the process of releasing and collecting assignments for the instructor and of fetching and submitting assignments for the student. In addition to using the nbgrader release_assignment, nbgrader fetch_assignment, nbgrader submit, and nbgrader collect commands (see Exchanging assignment files) with a shared server setup like JupyterHub, the formgrader (see Creating and grading assignments) can be configured to integrate with JupyterHub so that all grading can occur on the same server.

Warning

When using nbgrader with JupyterHub, it is strongly recommended to set a logfile so that you can more easily debug problems. To do so, you can set a config option, for example NbGrader.logfile = "/usr/local/share/jupyter/nbgrader.log".

Each of these use cases also has a corresponding demo in the demos folder of the GitHub repository.

Example Use Case: One Class, One Grader¶

The formgrader should work out-of-the-box with JupyterHub if you only have a single grader for your class: all you need to do is make sure that you have installed and enabled the nbgrader extensions (see Installation) and then make sure the path to your course directory is properly set in the instructor’s nbgrader_config.py. For example, if the instructor account is called instructor and your course directory is located in /home/instructor/course101/, then you should have a file at /home/instructor/.jupyter/nbgrader_config.py with contents like:

c = get_config()
c.CourseDirectory.root = '/home/instructor/course101'

The following figure describes the relationship between the instructor account, the student accounts, and the formgrader on JupterHub. In particular, note that in this case the formgrader is running directly on the instructor’s account:

Example Use Case: One Class, Multiple Graders¶

If you have multiple graders, then you can set up a shared notebook server as a JupyterHub service. I recommend creating a separate grader account (such as grader-course101) for this server to have access to.

The following figure describes the relationship between the instructor accounts, the student accounts, and the formgrader on JupterHub. In particular, note that in this case the formgrader is running as a separate service, which each instructor then has access to:

_images/one_class_multiple_instructors.png

You will additionally need to install and enable the various nbgrader extensions for different accounts (see Installation). This table should clarify which extension to enable when you have separate services for the formgraders.

	Students	Instructors	Formgraders
Create Assignment	no	no	yes
Assignment List	yes	yes	no
Formgrader	no	no	yes
Course List	no	yes	no

Your JupyterHub should look something like this:

c = get_config()

# Our user list
c.Authenticator.whitelist = [
    'instructor1',
    'instructor2',
    'student1',
]

# instructor1 and instructor2 have access to a shared server:
c.JupyterHub.load_groups = {
    'formgrade-course101': [
        'instructor1',
        'instructor2'
    ]
}

# Start the notebook server as a service. The port can be whatever you want
# and the group has to match the name of the group defined above. The name
# of the service MUST match the name of your course.
c.JupyterHub.services = [
    {
        'name': 'course101',
        'url': 'http://127.0.0.1:9999',
        'command': [
            'jupyterhub-singleuser',
            '--group=formgrade-course101',
            '--debug',
        ],
        'user': 'grader-course101',
        'cwd': '/home/grader-course101'
    }
]

Similarly to the use case with just a single grader, there needs to then be a nbgrader_config.py file in the root of the grader account, which points to the directory where the class files are, e.g. in /home/grader-course101/.jupyter/nbgrader_config.py:

c = get_config()
c.CourseDirectory.root = '/home/grader-course101/course101'

You will additionally need to add a global nbgrader config file (for example, in /etc/jupyter/nbgrader_config.py) which specifies the course id:

c = get_config()
c.CourseDirectory.course_id = 'course101'

This course id MUST match the name of the service that is running the formgrader.

Example Use Case: Multiple Classes¶

As in the case of multiple graders for a single class, if you have multiple classes on the same JupyterHub instance, then you will need to create multiple services (one for each course) and corresponding accounts for each service (with the nbgrader extensions enabled, see Installation). For example, you could have users grader-course101 and grader-course123 which access services called course101 and course123, respectively.

The following figure describes the relationship between the instructor accounts and the formgraders on JupterHub (student accounts are not shown, but are the same as in Example Use Case: One Class, Multiple Graders). In particular, note that in this case each formgrader is running as a separate service, which some combination of instructors then have access to:

JupyterHub Authentication¶

New in version 0.6.0.

With the advent of JupyterHubAuthPlugin, nbgrader will ask JupyterHub which students are enrolled in which courses and only show them assignments from those respective courses (note that the JupyterHubAuthPlugin requires JupyterHub version 0.8 or higher). Similarly, nbgrader will ask JupyterHub which instructors have access to which courses and only show them formgrader links for those courses.

On the JupyterHub side of things, to differentiate student from instructor, groups need to be named formgrade-{course_id} for instructors and and grader accounts, and nbgrader-{course_id} for students. The course service additionally needs to have an API token set that is from a JupyterHub admin (see JupyterHub documentation).

As in the case of multiple graders for a single class, if you have multiple classes on the same JupyterHub instance, then you will need to create multiple services (one for each course) and corresponding accounts for each service (with the nbgrader extensions enabled, see Installation). For example, you could have users grader-course101 and grader-course123. Your JupyterHub config would then look something like this:

c = get_config()

# Our user list
c.Authenticator.whitelist = [
    'instructor1',
    'instructor2',
    'student1',
    'grader-course101',
    'grader-course123'
]

c.Authenticator.admin_users = {
    'instructor1',
    'instructor2'
}

# instructor1 and instructor2 have access to different shared servers:
c.JupyterHub.load_groups = {
    'formgrade-course101': [
        'instructor1',
        'grader-course101',
    ],
    'formgrade-course123': [
        'instructor2',
        'grader-course123'
    ],
    'nbgrader-course101': [],
    'nbgrader-course123': []
}

# Start the notebook server as a service. The port can be whatever you want
# and the group has to match the name of the group defined above.
c.JupyterHub.services = [
    {
        'name': 'course101',
        'url': 'http://127.0.0.1:9999',
        'command': [
            'jupyterhub-singleuser',
            '--group=formgrade-course101',
            '--debug',
        ],
        'user': 'grader-course101',
        'cwd': '/home/grader-course101',
        'api_token': ''  # include api token from admin user
    },
    {
        'name': 'course123',
        'url': 'http://127.0.0.1:9998',
        'command': [
            'jupyterhub-singleuser',
            '--group=formgrade-course123',
            '--debug',
        ],
        'user': 'grader-course123',
        'cwd': '/home/grader-course123',
        'api_token': ''  # include api token from admin user
    },
]

Note: As you can see the nbgrader-{course_id} group is an empty list. Adding students to the JupyterHub group is automatically done when the instructor adds them to the course database with the nbgrader db student add command or through the formgrader.

On the nbgrader side of things, activating the JupyterHubAuthPlugin requires you to add it as an authentication plugin class into the nbgrader_config.py for all accounts. This is easiest to do by putting it in a global location such as /etc/jupyter/nbgrader_config.py. You also need to configure nbgrader to look for assignments in a subdirectory corresponding to the course name (see Can I use the “Assignment List” extension with multiple classes?). For example:

from nbgrader.auth import JupyterHubAuthPlugin
c = get_config()
c.Exchange.path_includes_course = True
c.Authenticator.plugin_class = JupyterHubAuthPlugin

There also needs to be a separate nbgrader_config.py file in the root of each grader account, which points to the directory where the class files are and which specifies what the course id is, e.g. /home/grader-course101/.jupyter/nbgrader_config.py would be:

c = get_config()
c.CourseDirectory.root = '/home/grader-course101/course101'
c.CourseDirectory.course_id = 'course101'

and /home/grader-course123/.jupyter/nbgrader_config.py would be:

c = get_config()
c.CourseDirectory.root = '/home/grader-course123/course123'
c.CourseDirectory.course_id = 'course123'

Finally, you will again need to enable and disable different combinations of the nbgrader extensions for different accounts. See the table in Example Use Case: One Class, Multiple Graders for details.

Custom Authentication¶

New in version 0.6.0.

To make your own custom authentication such as through an LTI you could start by making a method that inherits the Authenticator class, which is a plugin for different authentication methods.

There are now four authentication classes:

BaseAuthPlugin: Inherit this class when implementing your own plugin, thought of as a way to enable LTI use cases. This class is never called directly.
NoAuthPlugin: The default old behaviour. Using this plugin will allow any user to any course if they do not have a course_id in their nbgrader_config. This is still the default behaviour so no need to specify it in /etc/jupyter/nbgrader_config.py
JupyterHubAuthPlugin: Uses the Jupyterhub groups part of the JupyterHub API for authentication.
Authenticator: Configurable for different plugins.

API¶

class nbgrader.auth.BaseAuthPlugin(**kwargs)[source]¶

get_student_courses(student_id)[source]¶

Gets the list of courses that the student is enrolled in.

Parameters

student_id (str) – The unique id of the student.

Return type

Optional[list]

Returns

A list of unique course ids, or None. If None is returned this means
that the student has access to any course that might exist. Otherwise
the student is only allowed access to the specific courses returned in
the list.

add_student_to_course(student_id, course_id)[source]¶

Grants a student access to a given course.

Parameters

student_id (str) – The unique id of the student.
course_id (str) – The unique id of the course.

Return type

None

remove_student_from_course(student_id, course_id)[source]¶

Removes a student’s access to a given course.

Parameters

student_id (str) – The unique id of the student.
course_id (str) – The unique id of the course.

Return type

None

nbgrader and its exchange service¶

A simplistic overview¶

Assignments are created, generated, released, fetched, submitted, collected, graded. Then feedback can be generated, released, and fetched.

The exchange is responsible for recieving released assignments, allowing those assignments to be fetched, accepting submissions, and allowing those submissions to be collected. It also allows feedback to be transferred.

Calling exchange classes¶

Exchange functions are called three ways:

From the command line - eg: nbgrader release_assignment assignment1.
From formgrader server_extension, which generally calls the methods defined in nbgrader/apps/{foo}app.py.
From the assignment_list server_extension, which generally calls the methods directly.

How nbgrader server_extenstions call the exchange¶

Defined directories¶

CourseDirectory defines the following directories (and their defaults):

source_directory - Where new assignments that are created by instructors are put (defaults to source)
release_directory - Where assignments that have been processed for release are copied to (defaults to release)
submitted_directory - Where student submissions are copied to, when an instructor collects (defaults to submitted)
autograded_directory - Where student submissions are copied to, having been autograded (defaults to autograded)
feedback_directory - Where feedback is copied to, when Instructors generate feedback (defaults to feedback)
solution_directory - Where solution is copied to, when Istructors generate solution (defaults to solution)

Also, taken from the nbgrader help:

The nbgrader application is a system for assigning and grading notebooks.
Each subcommand of this program corresponds to a different step in the
grading process. In order to facilitate the grading pipeline, nbgrader
places some constraints on how the assignments must be structured. By
default, the directory structure for the assignments must look like this:

    {nbgrader_step}/{student_id}/{assignment_id}/{notebook_id}.ipynb

where 'nbgrader_step' is the step in the nbgrader pipeline, 'student_id'
is the ID of the student, 'assignment_id' is the name of the assignment,
and 'notebook_id' is the name of the notebook (excluding the extension).

`Exchange`¶

Base class. Contains some required configuration parameters and elements - the prominant ones include path_includes_course and coursedir.

This class defines the following methods which are expeceted to be overridden in subclasses:

init_src(): Define the location files are copied from
init_dest(): Define the location files are copied to
copy_files(): Actually copy the files.

The class also defines a convenience method, which may be overridden in subclasses:

def start(self):
    self.set_timestamp()
    self.init_src()
    self.init_dest()
    self.copy_files()

This method is used to perform the relevant action (fetch/release, list, submit etc.)

`ExchangeError`¶

This is the error that should be raised if any error occurs while performing any of the actions.

`ExchangeCollect`¶

Fetches [all] submissions for a specified assignment from the exchange and puts them in the [instructors] home space.

The exchange is called thus:

self.coursedir.assignment_id = assignment_id
exchange = ExchangeFactory(config=config)
collect = exchange.Collect(
    coursedir=self.coursedir,
    authenticator=self.authenticator,
    parent=self)
try:
    collect.start()
except ExchangeError:
    self.fail("nbgrader collect failed")

The config object passed to the ExchangeFactory needs to contain the configuration specifying which concrete class to use for the ExchangeCollect abstract class.

Expected behaviours¶

The expected destination for collected files is {self.coursedir.submitted_directory}/{student_id}/{self.coursedir.assignment_id}
collect.update is a flag to indicate whether collected files should be replaced if a later submission is available. There is an assumption this defaults to True

`ExchangeFetch`¶

(Depreciated, use ExchangeFetchAssignment)

`ExchangeFetchAssignment`¶

Gets the named assignment & puts the files in the users home space.

The nbgrader server_extension calls it thus:

with self.get_assignment_dir_config() as config:
    try:
        config = self.load_config()
        config.CourseDirectory.course_id = course_id
        config.CourseDirectory.assignment_id = assignment_id

        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        fetch = exchange.FetchAssignment(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        fetch.start()
    .....

Returns…. nothing

Expected behaviours¶

The expected destination for files is {self.assignment_dir}/{self.coursedir.assignment_id} however if self.path_includes_course is True, then the location should be {self.assignment_dir}/{self.coursedir.course_id}/{self.coursedir.assignment_id}

self.coursedir.ignore is described as a:

List of file names or file globs.
Upon copying directories recursively, matching files and
directories will be ignored with a debug message.

This should be honoured.

In the default exchange, existing files are not replaced.

`ExchangeFetchFeedback`¶

This copies feedback from the exchange into the students home space.

The nbgrader server_extension calls it thus:

with self.get_assignment_dir_config() as config:
    try:
        config = self.load_config()
        config.CourseDirectory.course_id = course_id
        config.CourseDirectory.assignment_id = assignment_id

        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        fetch = exchange.FetchFeedback(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        fetch.start()
    .....

returns…. nothing

Expected behaviours¶

Files should be copied into a feedback directory in whichever directory ExchangeFetchAssignment deposited files.
Each submission should be copied into a feedback/{timestamp} directory, where timestamp is the timestamp from the timestamp.txt file generated during the submission.

When writing your own Exchange¶

You to need to consider stopping students from seeing each others submissions

`ExchangeList`¶

This class is responsible for determining what assignments are available to the user.

It has three flags to define various modes of operation:

self.remove=True: If this flag is set, the assignment files (as defined below) are removed from the exchange.
self.inbound=True or self.cached=True: These both refer to submitted assignments. The assignment_list plugin sets config.ExchangeList.cached = True when it queries for submitted notebooks.
neither: This is released (and thus fetched) assignments.

Note that CourseDirectory and Authenticator are defined when the server_sextension assignment_list calls the lister:

with self.get_assignment_dir_config() as config:
    try:
        if course_id:
            config.CourseDirectory.course_id = course_id

        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        lister = exchange.List(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        assignments = lister.start()
    ....

returns a List of Dicts - eg:

[
    {'course_id': 'course_2', 'assignment_id': 'car c2', 'status': 'released', 'path': '/tmp/exchange/course_2/outbound/car c2', 'notebooks': [{'notebook_id': 'Assignment', 'path': '/tmp/exchange/course_2/outbound/car c2/Assignment.ipynb'}]},
    {'course_id': 'course_2', 'assignment_id': 'tree c2', 'status': 'released', 'path': '/tmp/exchange/course_2/outbound/tree c2', 'notebooks': [{'notebook_id': 'Assignment', 'path': '/tmp/exchange/course_2/outbound/tree c2/Assignment.ipynb'}]}
]

The format and structure of this data is discussed in ExchangeList Date Return structure below.

Note¶

This gets called TWICE by the assignment_list server_extension - once for released assignments, and again for submitted assignments.

`ExchangeRelease`¶

(Depreciated, use ExchangeReleaseAssignment)

`ExchangeReleaseAssignment`¶

This should copy the assignment from the release location (normally {self.coursedir.release_directory}/{self.coursedir.assignment_id}) and copies it into the exchange service.

The class should check for the assignment existing (look in {self.coursedir.release_directory}/{self.coursedir.assignment_id}) before actually copying

The exchange is called thus:

exchange = ExchangeFactory(config=config)
release = exchange.ReleaseAssignment(
    coursedir=self.coursedir,
    authenticator=self.authenticator,
    parent=self)
try:
    release.start()
except ExchangeError:
    self.fail(``nbgrader release_assignment failed``)

returns…. nothing

`ExchangeReleaseFeedback`¶

This should copy all the feedback for the current assignment to the exchange.

Feedback is generated by the Instructor. From GenerateFeedbackApp:

Create HTML feedback for students after all the grading is finished.
This takes a single parameter, which is the assignment ID, and then (by
default) looks at the following directory structure:

    autograded/*/{assignment_id}/*.ipynb

from which it generates feedback the the corresponding directories
according to:

    feedback/{student_id}/{assignment_id}/{notebook_id}.html

The exchange is called thus:

exchange = ExchangeFactory(config=config)
release_feedback = exchange.ReleaseFeedback(
    coursedir=self.coursedir,
    authenticator=self.authenticator,
    parent=self)
try:
    release_feedback.start()
except ExchangeError:
    self.fail("nbgrader release_feedback failed")

returns….. nothing

`ExchangeSubmit`¶

This should copy the assignment from the user’s work space, and make it available for instructors to collect.

The exchange is called thus:

with self.get_assignment_dir_config() as config:
    try:
        config = self.load_config()
        config.CourseDirectory.course_id = course_id
        config.CourseDirectory.assignment_id = assignment_id
        coursedir = CourseDirectory(config=config)
        authenticator = Authenticator(config=config)
        exchange = ExchangeFactory(config=config)
        submit = exchange.Submit(
            coursedir=coursedir,
            authenticator=authenticator,
            config=config)
        submit.start()
    .....

The source for files to be submitted needs to match that in ExchangeFetchAssignment.

returns…. nothing

When writing your own Exchange¶

You to need to consider stopping students from seeing each others submissions
nbgrader functionality requires a file called timestamp.txt to be in the submission, containing the timestamp of that submission. The creation of this file is the responsibility of this class.
Whilst nothing is done as yet, the default exchange checks the names of submitted notebooks, and logs differences.
Submissions need to record student_id, as well as course_id & assignment_id
The default exchange copies files to both an inbound and cache store.

ExchangeList Date Return structure¶

As mentioned in the ExchangeList class documentation above, this data is returned as a List of Dicts.

The format of the Dicts vary depending on the type of assignments being listed.

Removed¶

Returns a list of assignments formatted as below (whether they are released or submitted), but with the status set to removed

Released & Submitted¶

The first step is to loop through a list of assignments (lets call each one a path) and get some basic data:

released

{course_id: xxxx, assignment_id: yyyy}

submitted

{course_id: xxxx, assignment_id: yyyy, student_id: aaaa, timestamp: ISO 8601}

We then add status and path information:

if self.inbound or self.cached:
    info['status'] = 'submitted'
    info['path'] = path  # ie, where it is in the exchange
elif os.path.exists(assignment_dir):
    info['status'] = 'fetched'
    info['path'] = os.path.abspath(assignment_dir)  # ie, where it in on the students home space.
else:
    info['status'] = 'released'
    info['path'] = path # again, where it is in the exchange

if self.remove:
    info['status'] = 'removed'
    # Note, no path - it's been deleted.

(assignment_dir is the directory in the students home space, so needs to take into account self.path_includes_course)

Next loop through all the notebooks in the path, and get some basic data:

nb_info = {'notebook_id': /name, less extension/, 'path': /path_to_file/}

If the notebook is info['status'] != 'submitted':

that’s all the data we have:

info['notebooks'].append(nb_info)

else, add feedback details for this notebook:

nb_info['has_local_feedback'] = _has_local_feedback()
nb_info['has_exchange_feedback'] = _has_exchange_feedback()
if nb_info['has_local_feedback']:
    nb_info['local_feedback_path'] = _local_feedback_path()
if nb_info['has_local_feedback'] and nb_info['has_exchange_feedback']:
    nb_info['feedback_updated'] = _exchange_feedback_checksum() != _local_feedback_checksum()
info['notebooks'].append(nb_info)

Having looped through all notebooks

If info['status'] == 'submitted', add feedback notes to the top-level assignment record:

info['has_local_feedback'] = _any_local_feedback()
info['has_exchange_feedback'] = _any_exchange_feedback()
info['feedback_updated'] = _any_feedback_updated()
if info['has_local_feedback']:
    info['local_feedback_path'] = os.path.join(
        assignment_dir, 'feedback', info['timestamp'])
else:
    info['local_feedback_path'] = None

Using an alternative exchange¶

By default, the exchange assumes all users are unique on the host, and uses folders and file-copys to manage the distribution of files.

nbgrader will, however, allow alternative mechanisms for distribution.

The default exchange¶

The exchange package is organised as follows:

exchange/
├── abc/
│   ├── collect.py
│   ├── ...
│   └── submit.py
├── default/
│   ├── collect.py
│   ├── ...
│   └── submit.py
└── exchange_factory.py

The exchange.abc package contains all the Abstract Base Classes that custom exchange packages need to implement. The exchange.default package is a default filesystem based implementation. The exchange_factory.py file contains the defintion for the ExchangeFactory class that is used to create instances of the exchange classes.

The nbgrader exchange uses the followng classes:

Exchange
ExchangeError
ExchangeCollect
ExchangeFetch
ExchangeFetchAssignment
ExchangeFetchFeedback
ExchangeList
ExchangeRelease
ExchangeReleaseAssignment
ExchangeReleaseFeedback
ExchangeSubmit

Of these ExchangeFetch and ExchangeRelease have both been deprecated and not configurable through the ExchangeFactory class.

Configuring a custom exchange¶

To configure a custom exchange, you simply set the relevant field in the ExchangeFactory class through traitlets configuration service.

For example, if we have installed a package called nbexchange, we add the following to the nbgrader_config.py file:

## A plugin for collecting assignments.
c.ExchangeFactory.collect = 'nbexchange.plugin.ExchangeCollect'
## A plugin for exchange.
c.ExchangeFactory.exchange = 'nbexchange.plugin.Exchange'
## A plugin for fetching assignments.
c.ExchangeFactory.fetch_assignment = 'nbexchange.plugin.ExchangeFetchAssignment'
## A plugin for fetching feedback.
c.ExchangeFactory.fetch_feedback = 'nbexchange.plugin.ExchangeFetchFeedback'
## A plugin for listing exchange files.
c.ExchangeFactory.list = 'nbexchange.plugin.ExchangeList'
## A plugin for releasing assignments.
c.ExchangeFactory.release_assignment = 'nbexchange.plugin.ExchangeReleaseAssignment'
## A plugin for releasing feedback.
c.ExchangeFactory.release_feedback = 'nbexchange.plugin.ExchangeReleaseFeedback'
## A plugin for submitting assignments.
c.ExchangeFactory.submit = 'nbexchange.plugin.ExchangeSubmit'

The fields in the ExchangeFactory are:

exchange - The base Exchange class that all other classes inherit from
fetch_assignment - The ExchangeFetchAssignment class
fetch_feedback - The ExchangeFetchFeedback class
release_assignment - The ExchangeReleaseAssignment class
release_feedback - The ExchangeReleaseFeedback class
list - The ExchangeList class
submit - The ExchangeSubmit class
collect - The ExchangeCollect class

Adding customization plugins¶

Limited support is offered for customizing the behavior of nbgrader through using predefined plugin methods or by added your own custom plugin. See the example usage below for the supported plugins and their API.

If you need further customized behavior please open an issue.

PRs are also welcome!

Example usage¶

Late submission plugin¶

Assigning a penalty to notebooks that were submitted late can be done using the method described below. The default behavior is not to assign any penalty. nbgrader will still compute how late each submission is.

For this to work, you must include a duedate for the assignment and then a timestamp.txt file in the folder for each submission with a single line containing a timestamp (e.g. 2015-02-02 14:58:23.948203 America/Los_Angeles). Then, when you run nbgrader autograde, nbgrader will record these timestamps into the database, compute how late each submission is and assign a late penalty (if specified). Also see the faq.

Predefined methods¶

To assign an overall notebook score of zero for any late submission you can include the following in the {course_directory}/nbgrader_config.py file:

c.LateSubmissionPlugin.penalty_method = 'zero'

Creating a plugin:¶

To add your own custom management of late submissions you can create a plugin class. For example the {course_directory}/late.py module that assigns a penalty of 1 point per hour late::

from __future__ import division
from nbgrader.plugins import BasePlugin


class SubMarks(BasePlugin):
    def late_submission_penalty(self, student_id, score, total_seconds_late):
        """Penalty of 1 mark per hour late"""
        hours_late = total_seconds_late / 3600
        return round(hours_late, 0)

The class must inherit from BasePlugin and the late_submission_penalty function API is described below. The module and class names are arbitrary, but need to be added to the {course_directory}/nbgrader_config.py file, for example::

c.AssignLatePenalties.plugin_class = 'late.SubMarks'

Note: the late.py module can be either located in the same directory as where you are running the nbgrader commands (which is most likely the root of your course directory), or you can place it in one of a number of locations on your system. These locations correspond to the configuration directories that Jupyter itself looks in; you can find out what these are by running jupyter --paths.

API¶

class nbgrader.plugins.latesubmission.LateSubmissionPlugin(**kwargs)[source]¶

Predefined methods for assigning penalties for late submission

late_submission_penalty(student_id, score, total_seconds_late)[source]¶

Return the late submission penalty based on the predefined method.

Parameters

student_id (str) – The unique id of the student
score (float) – The score the student obtained for the submitted notebook
total_seconds_late (float) – The total number of seconds the submitted notebook was late

Returns

penalty – The assigned penalty score (None for no assigned penalty)

Return type

float OR None

Export plugin¶

New in version 0.4.0.

Many instructors need to be able to get grades for their class out of the nbgrader database and into another format, such as a CSV file, a learning management system (LMS) like Canvas or Blackboard, etc. nbgrader comes with the capability to export grades to a CSV file, however you may want to customize this functionality for your own needs.

Creating a plugin¶

To add your own grade exporter you can create a plugin class that inherits from nbgrader.plugins.ExportPlugin. This class needs to only implement one method, which is the export() method (see below). Let’s say you create your own plugin in the myexporter.py file, and your plugin is called MyExporter. Then, on the command line, you would run:

nbgrader export --exporter=myexporter.MyExporter

which will use your custom exporter rather than the built-in CSV exporter. For an example of how to interface with the database, please see Getting information from the database.

API¶

class nbgrader.plugins.export.ExportPlugin(**kwargs)[source]¶

Base class for export plugins.

export(gradebook)[source]¶

Export grades to another format.

This method MUST be implemented by subclasses. Users should be able to pass the --to flag on the command line, which will set the self.to variable. By default, this variable will be an empty string, which allows you to specify whatever default you would like.

Parameters: gradebook (Gradebook) – An instance of the gradebook
Return type: None

ZipCollect plugins¶

New in version 0.5.0.

Extractor plugin¶

Extract archive (zip) files in the archive_directory. Archive files are extracted to the extracted_directory. Non-archive (zip) files found in the archive_directory are copied to the extracted_directory. Archive files will be extracted into their own sub-directory within the extracted_directory and any archive files within archives will also be extracted into their own sub-directory along the path.

Creating a plugin¶

To add your own extractor you can create a plugin class that inherits from ExtractorPlugin. This class needs to only implement one method, which is the extract() method (see below). Let’s say you create your own plugin in the myextractor.py file, and your plugin is called MyExtractor. Then, on the command line, you would run:

nbgrader zip_collect --extractor=myextractor.MyExtractor

which will use your custom extractor rather than the built-in one.

API¶

class nbgrader.plugins.zipcollect.ExtractorPlugin(**kwargs)[source]¶

Submission archive files extractor plugin for the ZipCollectApp. Extractor plugin subclasses MUST inherit from this class.

extract(archive_path, extracted_path)[source]¶

Extract archive (zip) files and submission files in the archive_directory. Files are extracted to the extracted_directory. Non-archive (zip) files found in the archive_directory are copied to the extracted_directory. This is the main function called by the ZipCollectApp for each archive file to be extracted.

Parameters

archive_path (str) – Absolute path to the archive_directory.
extracted_path (str) – Absolute path to the extracted_directory.

Return type

None

FileNameCollector plugin¶

Apply a named group regular expression to each filename received from the ZipCollectApp and return None if the file should be skipped or a dictionary that, at the very least, contains the student_id and file_id key value pairs; and optionally contains the timestamp key value pair, for example:

dict(
    file_id='problem1.ipynb',
    student_id='hacker',
    timestamp='2017-01-30 15:30:10 UCT'
)

For more information about named group regular expressions see https://docs.python.org/howto/regex.html

Note: file_id must contain the relative path to the assignment when collecting submission files in assignment sub-folders, for example:

dict(
    file_id='data/sample.txt',
    student_id='hacker',
    timestamp='2017-01-30 15:30:10 UCT'
)

Creating a plugin¶

To add your own collector you can create a plugin class that inherits from FileNameCollectorPlugin. This class needs to only implement one method, which is the collect() method (see below). Let’s say you create your own plugin in the mycollector.py file, and your plugin is called MyCollector. Then, on the command line, you would run:

nbgrader zip_collect --collector=mycollector.MyCollector

which will use your custom collector rather than the built-in one.

API¶

class nbgrader.plugins.zipcollect.FileNameCollectorPlugin(**kwargs)[source]¶

Submission filename collector plugin for the ZipCollectApp. Collect plugin subclasses MUST inherit from this class.

collect(submitted_file)[source]¶

This is the main function called by the ZipCollectApp for each submitted file. Note this function must also return a dictionary or None for sub-classed plugins.

Parameters

submitted_file (str) – Each submitted file in the extracted_directory (absolute path).

Returns

Collected data from the filename or None if the file should be skipped. Collected data is a dict of the form:

{
    file_id: file_id,  # MUST be provided
    student_id: student_id,  # MUST be provided
    timestamp: timestamp  # Can optional be provided
}

Note: file_id MUST include the the relative path to the assignment if you are collecting files in assignment sub-folders.

Return type

groupdict

Overview¶

Welcome. We’re thrilled you want to contribute to nbgrader.

This guide gives an overview of various parts of the project and what our expectations are for contributions.

In general, the guidelines for opening issues, submitting PRs, code style, testing, and documentation, are the same as the IPython contribution guidelines.

Developer installation¶

Getting the source code¶

The source files for nbgrader and its documentation are hosted on GitHub. To clone the nbgrader repository:

git clone https://github.com/jupyter/nbgrader
cd nbgrader

Installing and building nbgrader¶

nbgrader installs and builds with one command:

pip install -r dev-requirements.txt -e .

Installing notebook extensions¶

Previously this was done using the nbgrader extension install command. However, moving forward this is done using the jupyter nbextension and jupyter serverextension commands.

The nbextensions are Javascript/HTML/CSS so they require separate installation and enabling. The –symlink option is recommended since it updates the extensions whenever you update the nbgrader repository. The serverextension is a Python module inside nbgrader, so only an enable step is needed. To install and enable all the frontend nbextensions (assignment list, create assignment, and formgrader) along with the server extensions (assignment list and formgrader) run:

jupyter nbextension install --symlink --sys-prefix --py nbgrader
jupyter nbextension enable --sys-prefix --py nbgrader
jupyter serverextension enable --sys-prefix --py nbgrader

To work properly, the assignment list and formgrader extensions require both the nbextension and serverextension. The create assignment extension only has an nbextension part.

Installing Firefox Headless WebDriver¶

To run tests while developing nbgrader and its documentation, the Firefox headless webdriver must be installed. Please follow the Mozilla installation instructions to get Firefox properly setup on your system.

JSON Metadata Format¶

nbgrader relies on metadata stored in the JSON notebook source. When you create a notebook using the “Create Assignment” extension, this extension saves various keys into the metadata that it then relies on during the steps of nbgrader generate_assignment. The nbgrader generate_assignment command also adds new metadata to the notebooks, which is used by nbgrader validate and nbgrader autograde.

The metadata is always stored at the cell level, in the cell’s metadata field, under a dictionary called nbgrader. This makes the notebook source look like:

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "nbgrader": {
                    ...
                }
            },
            "source": ["an example cell\n"]
        },
        ... more cells ...
    ],
    ... other notebook information ...
}

Details about the metadata are given below.

Version 2¶

Version 2 is the same as Version 1 (see below), except that it also includes the following optional key:

cell_type¶

Added by nbgrader generate_assignment.

This is the original cell type for the cell, which is used by nbgrader validate to determine whether the cell type has been changed by the student.

Version 1¶

The metadata may contain the following keys:

schema_version¶: The version of the metadata schema. Defaults to 1.

grade¶

Added by the “Create Assignment” extension.

Whether the cell should be graded (which essentially means whether it gets a point value). This should be true for manually graded answer cells and for autograder test cells.

solution¶

Added by the “Create Assignment” extension.

Whether the cell is a solution cell (which essentially means whether students should put their answers in it or not). This should be true for manually graded answer cells and autograded answer cells.

locked¶

Added by the “Create Assignment” extension.

Whether nbgrader should prevent the cell from being edited. This should be true for autograder test cells and any other cells that are marked as locked/read-only in the create assignment interface.

grade_id¶

Added by the “Create Assignment” extension.

This is the nbgrader cell id so that nbgrader can track its contents, outputs, etc.

points¶

Added by the “Create Assignment” extension.

This is the number of points that a cell is worth. It should only be set if grade is also set to true. The number of points must be greater than or equal to zero.

checksum¶

Added by nbgrader generate_assignment.

This is the checksum of the cell’s contents that can then be used by nbgrader validate and nbgrader autograde to determine whether the student has edited the cell.

Pull requests¶

Development workflow¶

We follow the development workflow described in the IPython contribution guidelines and IPython contributing wiki.

Pull request submission checklist¶

Before making a pull request, please ensure you have:

Added or updated relevant documentation, if applicable.
Added new tests, if applicable. This is especially important if the pull request is a major change or a bugfix. For these pull requests, regression tests are needed.
Run the full test suite locally before submitting the pull request.

Testing¶

Before making a PR, please run the test suite locally and make sure everything passes.

We have automatic tests on Travis CI, but they take a long time to run and sometimes randomly time out, so it will be much more efficient if you can test locally first.

Running the full test suite¶

To run the full test suite, run the following command from the root of the repository:

python tasks.py tests

Running selective groups of tests¶

To run a selective group of tests you can use one of the following commands:

Command	Task
`python tasks.py tests --group=python`	Run tests only for the Python code
`python tasks.py tests --group=nbextensions`	Run tests only for the notebook extensions
`python tasks.py tests --group=docs`	Build the docs
`python tasks.py tests --group=all`	Same as `python tasks.py tests`

Using pytest to run a single test module¶

If you want to choose an even more specific subset of tests, you should run pytest directly. For example, to run only the tests for nbgrader generate_assignment:

pytest nbgrader/tests/apps/test_nbgrader_assign.py

Documentation¶

The source for the documentation can be found in the docs/source directory of this repository. These source files are a combination of ReStructured Text (rst) and Jupyter notebooks.

Editing source files¶

ReStructured Text: The rst files should be fairly straightforward to edit. Here is a quick reference of rst syntax. Some of the rst files also use Sphinx autodoc.
Jupyter Notebooks: The Jupyter notebooks are written in Python and should be written so that they are compatible with Python 3. If you need to reference another part of the documentation from a notebook, you will need to put that reference in a raw cell in the notebook, not a markdown cell.

Adding a new file to documentation¶

If you add a new file (either rst or ipynb) make sure to link to it from the relevant index.rst.

Additionally, if you are adding a new notebook in the user guide, please add the rst version of it to .gitignore.

Building documentation locally¶

If you have made changes to the user guide or other notebooks that need to be executed, please make sure you re-run all the documentation before committing. While the documentation gets built automatically on Read The Docs, the notebooks do not get execute by Read The Docs – they must be executed manually. However, executing the notebooks is easy to do!

Our docs are built with nbconvert, Pandoc, and Sphinx. To build the docs locally, run the following command:

python tasks.py docs

This will perform a few different steps:

The notebooks are executed and converted to rst or html (the actual documentation notebooks will be converted to rst, and example notebooks will be converted to html) using the build_docs.py script.
The command line documentation is automatically generated using the autogen_command_line.py and autogen_config.py scripts.
The rst files are converted to html using Sphinx.

After running python tasks.py docs, the resultant HTML files will be in docs/build/html. You can open these files in your browser to preview what the documentation will look like (note, however, that the theme used by Read The Docs is different from the default Sphinx theme, so the styling will look different).

Automatic builds¶

When a commit is made on the master branch, documentation is automatically built by Read The Docs and rendered at nbgrader.readthedocs.org.

JavaScript dependencies¶

For the time being, we are committing JavaScript dependencies to the nbgrader repository as that makes nbgrader installation much easier.

Adding or updating JavaScript libraries¶

If you need to add a new library, or update the version of a library, you will need to have npm installed.

To install npm on OS X, use Homebrew to install node (npm will be installed along with node):

brew update
brew install node

To install npm on Linux with apt-get, use:

apt-get update
apt-get install node
apt-get install npm

Modify the bower.json file in the root of the nbgrader repository and then run:

python tasks.py js

This will download and install the correct versions of the dependencies to nbgrader/server_extensions/formgrader/static/components. Usually, JavaScript libraries installed in this way include a lot of extra files (e.g. tests, documentation) that we don’t want to commit to the nbgrader repository. If this is the case, please add these files to the .gitignore file so these extra files are ignored and don’t get committed.

Modifying the Database¶

Sometimes new features require modifying the database, such as adding a new column. Implementing such changes is fine, but can cause compatibility problems for users who are using an older version of the database schema. To resolve this issue, we use alembic to handle database migrations.

To use alembic to migrate the database schema, first run the following command:

python -m nbgrader.dbutil revision -m "a description of the change"

This will create a file in the directory nbgrader/alembic/versions, for example nbgrader/alembic/versions/7685cbef311a_foo.py. You will now need to edit the upgrade() and downgrade() functions in this file such that they appropriately make the database changes. For example, to add a column called extra_credit to the grade table:

def upgrade():
    op.add_column('grade', sa.Column('extra_credit', sa.Float))

def downgrade():
    op.drop_column('grade', 'extra_credit')

Please see the alembic documentation for further details on how these files work. Additionally, note that you both need to update the database schema in nbgrader/api.py (this describes how to create new databases) as well as using alembic to describe what changes need to be made to old databases.

You can test whether the database migration works appropriately by running:

nbgrader db upgrade

on an old version of the database.

Prerequisites ¶

All PRs should be tagged with the appropriate milestone. Several of the releasing scripts depend on this being done.
You will need a GitHub API key: https://github.com/settings/tokens

Backport PRs ¶

If you are making a bugfix release onto an existing branch (e.g. 0.3.x) then you need to backport merged PRs from master onto this branch. To do this, check out the release branch (e.g. 0.3.x) and then run the backport_pr.py script from the tools directory.

First, use it to find the list of PRs that need to be backported (remember that you should have tagged all PRs with the appropriate milestone):

./tools/backport_pr.py 0.3.1

Second, use the script to backport the PRs one-by-one, in the order that they were merged into master, for example:

./tools/backport_pr.py 0.3.x 123

If a lot of code has changed between the release branch and master, the patch generated by the backport script may not apply. In this case, you will want to either (1) edit the patch to make it apply (e.g., by fixing line numbers) or (2) just apply the changes manually.

Update the changelog ¶

Ideally, the Changelog (located at nbgrader/docs/source/changelog.rst) should be updated as things are changed. In practice, this doesn’t always happen as people forget to include changes to the changelog in PRs. However, the changelog should be fully up-to-date at the time of a release. To ensure that it is up-to-date, filter PRs on GitHub by the current milestone and make sure to include all major changes. In addition, please include a full list of merged PRs, which can be found using the changelog.py script in the tools directory, for example:

./tools/changelog.py 0.3.0

This will print out the list of merged PRs for the given milestone, which should then be included in the changelog.

Note that if you are updating the changelog on a release branch (e.g. 0.3.x), then you will need to also make the relevant changes on master.

Get the list of contributors ¶

To get the list of contributors, you can use the contributor_list.py script in the tools directory, for example:

./tools/contributer_list.py 0.3.0

This will print out the list of users who have submitted issues and/or PRs. This list should then be included in the changelog.

Bump the version number ¶

The version number needs to be changed in the following files:

nbgrader/_version.py
nbgrader/nbextensions/assignment_list/main.js
nbgrader/nbextensions/course_list/main.js
nbgrader/nbextensions/validate_assignment/main.js

Rebuild the documentation ¶

Regenerate all the documentation for this release by running:

python tasks.py docs

Make sure the linkcheck passes, and commit the results.

Make a PR ¶

At this point, make a pull request with the changes you have made so far. Make sure all the tests pass. After this point, you shouldn’t need to make any more changes to the source: the remaining steps all have to do with building and releasing packages and creating branches/tags on GitHub.

Clean all untracked files ¶

Make sure there are no old files lying around the repository. First, see what needs to be cleaned:

git clean -ndx

After verifying this doesn’t include anything important, clean them for real:

git clean -fdx

Build and release the pip package ¶

To build the pip package, run the release.py script in the tools directory:

./tools/release.py

This will do a few things, including converting the README file to rst (so it will display correctly on PyPI) and building the source distribution. Afterwards, you can upload the package to PyPI with:

pip install -U twine
python -m twine upload dist/*

Create a git tag and possibly branch ¶

If this is a new major release, create a new .x branch. For example, if this is the 0.3.0 release, create a branch called 0.3.x.

Additionally, regardless of whether this is a major release, create a tag for the release. Release tags should be prefixed with v, for example, v0.3.0.

Create a release on GitHub ¶

After pushing the tag (and branch, if necessary) to GitHub, create the actual release on GitHub. To do this, go to https://github.com/jupyter/nbgrader/releases and click the button for “Draft a new release”. Choose the tag you just created and set the title as “nbgrader <tag>”, where “<tag>” is the name of the tag (e.g. v0.3.0). Put in the release notes, which should be pretty much the same as what is in the changelog.

Build and release the conda packages ¶

The conda recipe has been moved to a separate repository (“feedstock”) and now publishes nbgrader to the conda-forge channel automatically via CI. The conda forge buildbot should detect once you’ve created a tag and will automatically create a PR for the new release within a short period of time (might be up to an hour). Wait for this PR to happen, and then follow the instructions in the nbgrader-feedstock.

Change to development version ¶

Bump the version again, this time to development. For example, if the release was 0.3.0, then the new version should be 0.4.0.dev0. Remember that the version needs to be changed in these files:

nbgrader/_version.py
nbgrader/nbextensions/assignment_list/main.js
nbgrader/nbextensions/course_list/main.js
nbgrader/nbextensions/validate_assignment/main.js

Changelog¶

A summary of changes to nbgrader.

0.7.x¶

0.7.1¶

The following PRs were merged for the 0.7.1 milestone:

PR #1607: More informative error messages in ClearSolutions
PR #1598: Fix mathjax in formgrade templates
PR #1593: Pin traitlets dependency for 0.7.x
PR #1590: Ensure html files aren’t copied over from documentation
PR #1582: Trivial typo: “int the database”
PR #1579: Only add extra_template_basedirs if it has not been set
PR #1576: Revert “[converters/autograde] Fix autograded notebook permission”
PR #1518: [converters/autograde] Fix autograded notebook permission

Thanks to the following users who submitted PRs or reported issues that were merged or fixed for the 0.7.1 release:

Anmol23oct
brichet
jhamrick
kno10
mhwasil
szazs89
tmetzl

0.7.0¶

The following PRs were merged for the 0.7.0 milestone:

PR #1572: Fix a false positive test
PR #1571: Add workflow to enforce GitHub labels
PR #1569: Add Python 3.10 to CI pipeline
PR #1568: Update markupsafe requirement from <2.1.0 to <2.2.0
PR #1567: Upgrade nbconvert
PR #1565: Bump pytest from 6.2.4 to 7.1.2
PR #1564: Pin to notebook<7 for now
PR #1561: Add missing ‘self’ argument to late_submission_penalty
PR #1559: Fix breaking tests due to changes in the newest Jinja2 release
PR #1558: Bump pytest-xdist from 2.4.0 to 2.5.0
PR #1557: Update jupyter-client requirement from <7 to <8
PR #1541: Update setup.py with dependency ranges
PR #1539: Improve CI by running sphinx linkcheck
PR #1519: Make generate solutions preprocessors configurable
PR #1504: Bump sqlalchemy from 1.4.23 to 1.4.25
PR #1503: Bump pytest-xdist from 2.2.1 to 2.4.0
PR #1502: Bump alembic from 1.7.1 to 1.7.3
PR #1498: Bump rapidfuzz from 1.5.1 to 1.6.2
PR #1497: Bump notebook from 6.4.3 to 6.4.4
PR #1496: docs/index: Move setup-related topics to configuration section
PR #1494: docs: update highlights to introduce the notebook format
PR #1493: docs: revise “Managing assignment files” pages
PR #1489: Bump rapidfuzz from 1.4.1 to 1.5.1
PR #1488: Bump traitlets from 5.0.5 to 5.1.0
PR #1487: Bump alembic from 1.6.5 to 1.7.1
PR #1480: Bump sqlalchemy from 1.4.22 to 1.4.23
PR #1478: Bump notebook from 6.4.2 to 6.4.3
PR #1477: Bump notebook from 6.4.1 to 6.4.2
PR #1476: Fix Issue with Courses tab on Multi courses
PR #1475: Bump notebook from 6.4.0 to 6.4.1
PR #1472: Bump sqlalchemy from 1.4.21 to 1.4.22
PR #1470: Update badges in README
PR #1469: Bump python-dateutil from 2.8.1 to 2.8.2
PR #1468: Bump sqlalchemy from 1.4.20 to 1.4.21
PR #1467: Bump sqlalchemy from 1.4.18 to 1.4.20
PR #1466: Bump requests from 2.25.1 to 2.26.0
PR #1458: Lock setup dependencies
PR #1457: Add missing rollbacks to try/except clauses that execute db commits
PR #1450: Update autograding_resources.rst
PR #1444: Remove continuous integration for python 3.6
PR #1442: Bump traitlets from 4.3.3 to 5.0.5
PR #1441: Update pytest requirement from <6.0.0,>=4.5 to 6.2.4
PR #1440: Bump pytest-xdist from 1.34.0 to 2.2.1
PR #1438: Validate pre and post convert hooks
PR #1437: Make converter exporter class configurable
PR #1431: Add dependabot configuration
PR #1425: Use NBGRADER_VALIDATING env var during autograding
PR #1422: Fix docs building
PR #1420: Fix various SQLAlchemy errors and warnings
PR #1419: Update releasing docs and tools
PR #1394: Added CLI for generating solution notebooks
PR #1381: find cell failure when stderr is used
PR #1376: Make preprocessors of generate assignment, autograde and generate feedback configurable
PR #1330: Update azure pipelines matrix to add Python 3.8
PR #1329: Update the test matrix on Travis to Python 3.6+
PR #1324: Ensure errors are written to cell outputs to prevent the autograder from awarding points for failed tests
PR #1320: Add nbgrader collect –before-duedate option
PR #1315: ExchangeFetchAssignment deleting the wrong config
PR #1287: Add mypy for type checking
PR #1282: Further type annotations across the codebase
PR #1276: remove db_assignments db_students
PR #1274: Further Python 3 type annotations on top-level files
PR #1268: Type annotations for the api
PR #1259: Remove Python 2 compatibility code
PR #1257: Deprecate Python 2 support
PR #1238: Pluggable exchange
PR #1222: CourseDir.format_path: supports absolute paths in nbgrader_step

Thanks to the following users who submitted PRs or reported issues that were merged or fixed for the 0.7.0 release:

aliniknejad
AnotherCodeArtist
bbhopesh
BertR
brichet
elesiuta
gymreklab
HanTeo
jgwerner
jhamrick
jnishii
jtpio
LaurentHayez
liuq
lzach
nthiery
omelnikov
QuantumEntangledAndy
rkdarst
ryanlovett
samarthbhargav
sigurdurb
Tebinski
tmetzl
Wildcarde
willingc
ykazakov

0.6.x¶

0.6.2¶

nbgrader version 0.6.2 is a bugfix release. The following PRs were merged:

PR #1443: Fix broken windows tests
PR #1410: partial credit returns zero when score is zero
PR #1388: Move from travis ci to github actions
PR #1384: Fix migrations.
PR #1369: Pin nbconvert to 5.6.1, traitlets to 4.3.3 and pytest to <6.0.0
PR #1362: Fix migration, grade cells were looking for a non-existing column
PR #1356: add SAS codestub and autograde for metakernel based non-python kernels
PR #1352: Description of “what is nbgrader?”
PR #1343: Update deprecated jquery functions and update jquery
PR #1341: Make format_path behave the same for absolute paths
PR #1319: use rapidfuzz instead of fuzzywuzzy
PR #1308: docs: Fix formgrader group name in docs
PR #1288: Fixes #1283: Replace AppVeyor badge with Azure Devops badge
PR #1281: Demos using Python3
PR #1249: timestamp_format raises an exception

Thanks to the following users who submitted PRs or reported issues that were merged or fixed for the 0.6.1 release:

BertR
chinery
echuber2
enisnazif
fredcallaway
HanTeo
jgwerner
jhamrick
jld23
kcranston
lzach
maxbachmann
nklever
Patil2099
rkdarst
tmetzl

0.6.1¶

nbgrader version 0.6.1 is a bugfix release. The following PRs were merged:

PR #1280: Fix inappropriate use of sum with newer sqlite
PR #1278: Fix course list hanging when exchange has not been created
PR #1272: Improve test coverage in auth folder
PR #1270: Add requirements for readthedocs
PR #1267: Improve the error message on the assignments page
PR #1260: Set up CI with Azure Pipelines
PR #1245: Move away from using the internal Traitles API to load default configuration.
PR #1243: Fix project name typo
PR #1228: Fix formgrader API
PR #1227: Bump pytest required version to 4.5 for custom marker support
PR #1208: Improve coverage of nbgraderformat
PR #1205: Check for newer feedback in nbgrader list
PR #1204: Force generate feedback by default in API
PR #1200: Associate feedback files with unique submission attempts
PR #1197: Do not duplicate assignments when fetching feedback
PR #1196: Fix config warning in ExchangeReleaseAssignment
PR #1194: Update releasing instructions

Thanks to the following users who submitted PRs or reported issues that were merged or fixed for the 0.6.1 release:

BertR
enisnazif
jhamrick
kinow
nthiery
sir-dio

0.6.0¶

nbgrader version 0.6.0 is a major release, involving over 100 PRs and 60 issues. This includes many bug fixes, small enhancements, and improved docs. The major new features include:

Better support for multiple classes with JupyterHub. In particular, a new “Course List” extension has been added which provides instructors access to separate formgrader instances for all the classes they can manage. Additionally, JupyterHub authentication is used to control which students have access to which assignments.
Better LMS integration (for example, adding a lms_user_id column in the Student table of the database).
Better support for feedback. In particular, there is now the ability to generate and return feedback to students through nbgrader with the generate_feedback and release_feedback commands, and the ability for students to fetch feedback with the fetch_feedback command. This functionality is also available through the formgrader and Assignment List extensions.
Instructions for how to do grading inside a Docker container, for increased protection against malicious code submitted by students.
A new type of nbgrader cell called a “task” cell which supports more open-ended solutions which may span multiple cells.

Important: Users updating from 0.5.x to 0.6.0 should be aware that they will need to do the following (please make sure to back up your files before doing so, just in case anything goes wrong!):

Update their nbgrader database using nbgrader db upgrade.
Update the metadata in their assignments using nbgrader update.
Reinstall the nbgrader extensions (see Installation).

Please also note that some of the nbgrader commands have been renamed, for consistency with the new feedback commands:

nbgrader assign is now nbgrader generate_assignment
nbgrader release is now nbgrader release_assignment
nbgrader fetch is now nbgrader fetch_assignment

The full list of PRs is:

PR #1191: Allow access to formgrader when not using JuptyerHub auth
PR #1190: Add JupyterHub demos
PR #1186: Remove student_id and change root to cache, permission check to only execute
PR #1184: Move the fetch feedback API from formgrader to assignment_list
PR #1183: Feedback: update fetch_feedback command line help
PR #1180: Fix versions of pytest and nbconvert
PR #1179: Add CourseDir.student_id_exclude option to exclude students
PR #1169: Fix minor typo in js extension helper text
PR #1164: assignment_dir: Add into several missing places
PR #1152: Rename ‘nbgrader fetch’ to ‘nbgrader fetch_assignment’
PR #1151: Rename ‘nbgrader release’ to ‘nbgrader release_assignment’
PR #1147: Add test to ensure that db upgrade succeeds before running assign
PR #1145: Rename nbgrader feedback to nbgrader generate_feedback
PR #1140: A few more updates to the docs for multiple classes
PR #1139: Additional docs sanitization
PR #1138: Ensure that cell type changes result in valid nbgrader metadata
PR #1137: Rename “nbgrader assign” to “nbgrader generate_assignment”
PR #1135: section on grading in docker container
PR #1131: Better support for multiple classes
PR #1127: Better documentation of nbgrader_config.py
PR #1126: Remove the third party resources page
PR #1125: Check that the course directory is a subdirectory of the notebook dir
PR #1124: Only run nbextensions tests on oldest and newest versions of python
PR #1123: Ensure course directory root path has no trailing slashes
PR #1122: Fix incorrect usage of Exchange.course_id
PR #1121: Fix logfile
PR #1120: Integrate feedback distribution within nbgrader
PR #1119: added a sanatizing step to the doc creation.
PR #1118: Integrate course_id into the api and apps
PR #1116: Autograde & Assign: create missing students/assignments by default
PR #1115: Fix typo in tmp filename prefix in conftest.py
PR #1114: Documentation for multiple classes
PR #1113: Add a course list extension that shows all courses an instructor can manage
PR #1112: Locate all configurable classes for generate_config subcommand
PR #1111: Optional consistency check between owner and student_id upon collect
PR #1110: Systematic use of utils.get_username instead of $USER
PR #1109: naming the temporary directories in tests
PR #1108: Extended support for filtering files copied in the exchange
PR #1106: Remove testing of python 3.4
PR #1105: Remove extra keys in nbgrader metadata and better schema mismatch errors
PR #1102: Only build docs with one version of python
PR #1101: Add jupyter education book to third party resources
PR #1100: Run test in the python group in parallel using pytest-xdist
PR #1099: Add course table, add course_id column to assignment
PR #1098: Customizable student ID in nbgrader submit
PR #1094: Update license
PR #1093: Add authentication plugin support
PR #1090: partial credit for autograde test cells
PR #1088: Remove version requirement from urllib3
PR #1084: Fix miscellaneous bugs
PR #1080: compatibility with SQLAlchemy 1.3+
PR #1075: Give ExecutePreprocessor the Traitlets config during validation
PR #1071: student and assignment selection in exportapp implemented
PR #1064: Validate all cells
PR #1061: Set env var NBGRADER_VALIDATING when validating
PR #1054: Raise error when executed task fails
PR #1053: Remove changes to sitecustomize.py and dependency on invoke
PR #1051: Remove spellcheck and enchant dependency
PR #1040: Restrict access for students to different courses
PR #1036: Add a general lms user id column to the student table
PR #1032: fix: return info of reper function is wrong in api.py
PR #1029: Documentation fix to add info re: timeout errors.
PR #1028: Some improvements to the contributor list script
PR #1026: Mark test_same_part_navigation as flaky
PR #1025: Fixing failing tests, take 2
PR #1024: Fix deprecation warning with timezones
PR #1023: Ensure nbgrader list still works with random strings
PR #1021: Fix tests, all of which are failing :(
PR #1019: Make nbgrader quickstart work with existing directories
PR #1018: Add missing close > for url to display correctly
PR #1017: Fix all redirection
PR #1014: a mistake in comment
PR #1005: Add random string to submission filenames for better hiding
PR #1002: Change to notebook directory when validating (repeat of #880)
PR #1001: Allow setting a different assignment dir for students than the root notebook directory
PR #1000: Allow instructors to share files via shared group id
PR #994: Add link to jupyter in education map
PR #991: Fix broken documentation
PR #990: Include section on mocking (autograding resources)
PR #989: Update developer installation instructions
PR #984: Adding global graded tasks
PR #975: Fix the link to the activity magic
PR #972: Use mathjax macro for formgrader
PR #967: Added note in FAQ about changing cell ids
PR #964: Added “if __name__ == “__main__”:”
PR #963: Add third party resources to the documentation
PR #962: Add grant_extension method to the gradebook
PR #959: Allow apps to use -f and –force
PR #958: Do some amount of fuzzy problem set name matching
PR #957: Remove underscores from task names
PR #955: Ignore .pytest_cache in .gitignore
PR #954: Fix bug in find_all_files that doesn’t properly ignore directories
PR #953: update log.warn (deprecated) to log.warning
PR #948: Move config file generation to a separate app
PR #947: Exclude certain assignment files from being overwritten during autograding
PR #946: Fix failing tests
PR #937: Strip whitespace from assignment, student, and course ids
PR #936: Switch from PhamtomJS to Firefox
PR #934: Skip filtering notebooks when ExchangeSubmit.strict == True
PR #933: Fix failing tests
PR #932: Prevent assignments from being created with invalid names
PR #911: Update installation.rst
PR #909: Friendlier error messages when encountering a schema mismatch
PR #908: Better validation errors when cell type changes
PR #906: Resolves issues with UTF-8
PR #905: Update changelog and rebuild docs from 0.5.4
PR #900: Improve issue template to explain logic behind filling it out
PR #899: Help for csv import
PR #897: Give more details on how to use formgrader and jupyterhub
PR #892: Format code blocks in installation instructions
PR #886: Add nbval for non-Windows tests/CI
PR #877: Create issue_template.md
PR #871: Fix NbGraderAPI.timezone handling
PR #870: added java, matlab, and octave codestubs to clearsolutions.py
PR #853: Update changelog from 0.5.x releases
PR #838: Fetch multiple assignments in one command

Huge thanks to the following users who submitted PRs or reported issues that were merged or fixed for the 0.6.0 release:

00Kai0
Alexanderallenbrown
aliandra
amellinger
BertR
Carreau
cdvv7788
Ciemaar
consideRatio
damianavila
danielmaitre
DavidNemeskey
davidpwilliamson
davis68
ddbourgin
ddland
dechristo
destitutus
dsblank
edouardtheron
fenwickipedia
fm75
FranLucchini
gertingold
hcastilho
JanBobolz
jedbrown
jhamrick
jnak12
kcranston
kthyng
lgpage
liffiton
mikezawitkowski
mozebdi
mpacer
nabriis
nthiery
perllaghu
QuantumEntangledAndy
rgerkin
rkdarst
Ruin0x11
rwest
ryanlovett
samhinshaw
Sefriol
sigurdurb
slel
soldis
swarnava
takluyver
thotypous
vahtras
VETURISRIRAM
vidartf
willingc
yangkky
zonca

0.5.x¶

0.5.6¶

nbgrader version 0.5.6 is a small release that only unpins the version of IPython and Jupyter console.

0.5.5¶

nbgrader version 0.5.5 is a release for the Journal of Open Source education, with the following PRs merged:

PR #1057: Ensure consistency in capitalizing Jupyter Notebook
PR #1049: Update test builds on Travis
PR #1047: JOSE paper bib updates
PR #1045: Dev requirements and spelling tests
PR #1016: Fix anaconda link
PR #973: Create a paper on nbgrader

Thanks to the following users who submitted PRs or reported issues that were fixed for the 0.5.5 release:

jedbrown
jhamrick
swarnava
willingc

0.5.4¶

nbgrader version 0.5.4 is a bugfix release, with the following PRs merged:

PR #898: Make sure validation is run in the correct directory
PR #895: Add test and fix for parsing csv key names with spaces
PR #888: Fix overwritekernelspec preprocessor and update tests
PR #880: change directory when validating notebooks
PR #873: Fix issue with student dictionaries when assignments have zero points

Thanks to the following users who submitted PRs or reported issues that were fixed for the 0.5.4 release:

jcsutherland
jhamrick
lgpage
misolietavec
mpacer
ncclementi
randy3k

0.5.3¶

nbgrader version 0.5.3 is a bugfix release, with the following PRs merged:

PR #868: Fix travis to work with trusty
PR #867: Change to the root of the course directory before running nbgrader converters
PR #866: Set nbgrader url prefix to be relative to notebook_dir
PR #865: Produce warnings if the exchange isn’t set up correctly
PR #864: Fix link to jupyterhub docs
PR #861: fix the html to ipynb in docs

Thanks to the following users who submitted PRs or reported issues that were fixed for the 0.5.3 release:

jhamrick
misolietavec
mpacer
rdpratti

0.5.2¶

nbgrader version 0.5.2 is a bugfix release, with most of the bugs being discovered and subsequently fixed by the sprinters at SciPy 2017! The following PRs were merged:

PR #852: Fix spelling wordlist, again
PR #850: Include extension with feedback template filename
PR #848: Add links to the scipy talk
PR #847: Fix html export config options to avoid warnings
PR #846: Disallow negative point values
PR #845: Don’t install assignment list on windows
PR #844: Reveal ids if names aren’t set
PR #843: Update spelling wordlist
PR #840: Avoid extension errors when exchange is missing
PR #839: Always raise on convert failure
PR #837: Report mismatch extension versions
PR #836: Add documentation for course_id and release
PR #835: DOC: correct Cell Toolbar location
PR #833: Include quickstart .ipynb header
PR #831: Fix typo on Managing assignment docs
PR #830: Print out app subcommands by default
PR #825: Add directory structure example
PR #824: Add FAQ sections
PR #823: Typo fix.
PR #819: Update install instructions
PR #816: Add jupyter logo
PR #802: Fix bug with autograding when there is no timestamp

Thanks to the following users who submitted PRs or reported issues that were fixed for the 0.5.2 release:

arcticbarra
BjornFJohansson
hetland
ixjlyons
jhamrick
katyhuff
ksunden
lgpage
ncclementi
Ruin0x11

0.5.1¶

nbgrader version 0.5.1 is a bugfix release mainly fixing an issue with the formgrader. The following PRs were merged:

PR #792: Make sure relative paths to source and release dirs are correct
PR #791: Use the correct version number in the docs

0.5.0¶

nbgrader version 0.5.0 is another very large release with some very exciting new features! The highlights include:

The formgrader is now an extension to the notebook, rather than a standalone service.
The formgrader also includes functionality for running nbgrader assign, nbgrader release, nbgrader collect, and nbgrader autograde directly from the browser.
A new command nbgrader zip_collect, which helps with collecting assignment files downloaded from a LMS.
Hidden test cases are now supported.
A lot of functionality has moved into standalone objects that can be called directly from Python, as well as a high-level Python API in nbgrader.apps.NbGraderAPI (see High-Level API).
A new Validate notebook extension, which allows students to validate an assignment notebook from the notebook itself (this is equivalent functionality to the “Validate” button in the Assignment List extension, but without requiring students to be using the Assignment List).
A new command nbgrader db upgrade, which allows you to migrate your nbgrader database to the latest version without having to manually execute SQL commands.
New cells when using the Create Assignment extension will automatically given randomly generated ids, so you don’t have to set them yourself.
You can assign extra credit when using the formgrader.

Important: Users updating from 0.4.x to 0.5.0 should be aware that they will need to update their nbgrader database using nbgrader db upgrade and will need to reinstall the nbgrader extensions (see Installation). Additionally, the configuration necessary to use the formgrader with JupyterHub has changed, though it is now much more straightforward (see Using nbgrader with JupyterHub).

The full list of merged PRs includes:

PR #789: Fix more inaccurate nbextension test failures after reruns
PR #788: Fix inaccurate nbextension test failures after reruns
PR #787: Fix slow API calls
PR #786: Update documentation for nbgrader as a webapp
PR #784: Fix race condition in validate extension tests
PR #782: Implement nbgrader as a webapp
PR #781: Assign missing notebooks a score of zero and mark as not needing grading
PR #780: Create a new high-level python API for nbgrader
PR #779: Update the year!
PR #778: Create and set permissions for exchange directory when using nbgrader release
PR #774: Add missing config options
PR #772: Standalone versions of nbgrader assign, autograde, and feedback
PR #771: Fix mathjax rendering
PR #770: Better cleanup when nbconvert-based apps crash
PR #769: Fix nbgrader validate globbing for real this time
PR #768: Extra credit
PR #766: Make sure validation works with notebook globs
PR #764: Migrate database with alembic
PR #762: More robust saving of the notebook in create assignment tests
PR #761: Validate assignment extension
PR #759: Fix nbextension tests
PR #758: Set random cell ids
PR #756: Fix deprecations and small bugs
PR #755: Fast validate
PR #754: Set correct permissions when submitting assignments
PR #752: Add some more informative error messages in zip collect
PR #751: Don’t create the gradebook database until formgrader is accessed
PR #750: Add documentation for how to pass numeric ids
PR #747: Skip over students with empty submissions
PR #746: Fix bug with –to in custom exporters
PR #738: Refactor the filtering of existing submission notebooks for formgrader
PR #735: Add DataTables functionality to existing formgrade tables
PR #732: Fix the collecting of submission files for multiple attempts of multiple notebook assignments
PR #731: Reset late submission penalty before checking if submission is late or not
PR #717: Update docs regarding solution delimeters
PR #714: Preserve kernelspec when autograding
PR #713: Use new exchange functionality in assignment list app
PR #712: Move exchange functionality into non-application classes
PR #711: Move some config options into a CourseDirectory object.
PR #709: Fix formgrader tests link for 0.4.x branch (docs)
PR #707: Force rerun nbgrader commands
PR #704: Fix nbextension tests
PR #701: Set proxy-type=none in phantomjs
PR #700: use check_call for extension installation in tests
PR #698: Force phantomjs service to terminate in Linux
PR #696: Turn the gradebook into a context manager
PR #695: Use sys.executable when executing nbgrader
PR #693: Update changelog from 0.4.0
PR #681: Hide tests in “Autograder tests” cells
PR #622: Integrate the formgrader into the notebook
PR #526: Processing of LMS downloaded submission files

Thanks to the following contributors who submitted PRs or reported issues that were merged/closed for the 0.5.0 release:

AnotherCodeArtist
dementrock
dsblank
ellisonbg
embanner
huwf
jhamrick
jilljenn
lgpage
minrk
suchow
Szepi
whitead
ZelphirKaltstahl
zpincus

0.4.x¶

0.4.0¶

nbgrader version 0.4.0 is a substantial release with lots of changes and several new features. The highlights include:

Addition of a command to modify students and assignments in the database (nbgrader db)
Validation of nbgrader metadata, and a command to automatically upgrade said metadata from the previous version (nbgrader update)
Support for native Jupyter nbextension and serverextension installation, and deprecation of the nbgrader nbextension command
Buttons to reveal students’ names in the formgrader
Better reporting of errors and invalid submissions in the “Assignment List” extension
Addition of a menu to change between different courses in the “Assignment List” extension
Support to run the formgrader as an official JupyterHub service
More flexible code and text stubs when creating assignments
More thorough documentations

Important: Users updating from 0.3.x to 0.4.0 should be aware that they will need to update the metadata in their assignments using nbgrader update and will need to reinstall the nbgrader extensions (see Installation). Additionally, the configuration necessary to use the formgrader with JupyterHub has changed, though it is now much less brittle (see Using nbgrader with JupyterHub).

The full list of merged PRs includes:

PR #689: Add cwd to path for all nbgrader apps
PR #688: Make sure the correct permissions are set on released assignments
PR #687: Add display_data_priority option to GetGrades preprocessor
PR #679: Get Travis-CI to build
PR #678: JUPYTERHUB_SERVICE_PREFIX is already the full URL prefix
PR #672: Undeprecate –create in assign and autograde
PR #670: Fix deprecation warnings for config options
PR #665: Preventing URI Encoding of the base-url in the assignment_list extension
PR #656: Update developer installation docs
PR #655: Fix saving notebook in create assignment tests
PR #652: Make 0.4.0 release
PR #651: Update changelog with changes from 0.3.3 release
PR #650: Print warning when no config file is found
PR #649: Bump the number of test reruns even higher
PR #646: Fix link to marr paper
PR #645: Fix coverage integration by adding codecov.yml
PR #644: Add AppVeyor CI files
PR #643: Add command to update metadata
PR #642: Handle case where points is an empty string
PR #639: Add and use a Gradebook contextmanager for DbApp and DbApp tests
PR #637: Update conda channel to conda-forge
PR #635: Remove conda recipe and document nbgrader-feedstock
PR #633: Remove extra level of depth in schema per @ellisonbg
PR #630: Don’t fail test_check_version test on 'import sitecustomize' failed error
PR #629: Update changelog for 0.3.1 and 0.3.2
PR #628: Make sure to include schema files
PR #625: Add “nbgrader db” app for modifying the database
PR #623: Move server extensions into their own directory
PR #621: Replace tabs with spaces in installation docs
PR #620: Document when needs manual grade is set
PR #619: Add CI tests for python 3.6
PR #618: Implement formgrader as a jupyterhub service
PR #617: Add ability to show student names in formgrader
PR #616: Rebuild docs
PR #615: Display assignment list errors
PR #614: Don’t be as strict about solution delimeters
PR #613: Update FAQ with platform information
PR #612: Update to new traitlets syntax
PR #611: Add metadata schema and documentation
PR #610: Clarify formgrader port and suppress notebook output
PR #607: Set instance variables in base auth class before running super init
PR #598: Conda recipe - nbextension link / unlink scripts
PR #597: Re-submitting nbextension work from previous PR
PR #594: Revert “Use jupyter nbextension/serverextension for installation/activation”
PR #591: Test empty and invalid timestamp strings
PR #590: Processing of invalid notebook_id
PR #585: Add catches for empty timestamp files and invalid timestamp strings
PR #581: Update docs with invoke test group commands
PR #571: Convert readthedocs links for their .org -> .io migration for hosted projects
PR #567: Handle autograding failures better
PR #566: Add support for true read-only cells
PR #565: Add option to nbgrader fetch for replacing missing files
PR #564: Update documentation pertaining to the assignment list extension
PR #563: Add ability to switch between courses in assignment list extension
PR #562: Add better support to transfer apps for multiple courses
PR #550: Add documentation regarding how validation works
PR #545: Document how to customize the student version of an assignment
PR #538: Use official HubAuth from JupyterHub
PR #536: Create a “nbgrader export” command
PR #523: Allow code stubs to be language specific

Thanks to the following contributors who submitted PRs or reported issues that were merged/closed for the 0.4.0 release:

adamchainz
AstroMike
ddbourgin
dlsun
dsblank
ellisonbg
huwf
jhamrick
lgpage
minrk
olgabot
randy3k
whitead
whositwhatnow
willingc

0.3.x¶

0.3.3¶

Version 0.3.3 of nbgrader is a minor bugfix release that fixes an issue with running nbgrader fetch on JupyterHub. The following PR was merged for the 0.3.3 milestone:

PR #600: missing sys.executable, “-m”, on fetch_assignment

Thanks to the following contributors who submitted PRs or reported issues that were merged/closed for the 0.3.3 release:

alikasamanli
hetland

0.3.2¶

Version 0.3.2 of nbgrader includes a few bugfixes pertaining to building nbgrader on conda-forge.

PR #608: Fix Windows tests
PR #601: Add shell config for invoke on windows
PR #593: Send xsrf token in the X-XSRF-Token header for ajax
PR #588: basename to wordslist
PR #584: Changes for Notebook v4.3 tests

Thanks to lgpage, who made all the changes necessary for the 0.3.2 release!

0.3.1¶

Version 0.3.1 of nbgrader includes a few bugfixes pertaining to PostgreSQL and updates to the documentation. The full list of merged PRs is:

PR #561: Close db engine
PR #548: Document how to install the assignment list extension for all users
PR #546: Make it clearer how to set due dates
PR #535: Document using JupyterHub with SSL
PR #534: Add advanced topics section in the docs
PR #533: Update docs on installing extensions

Thanks to the following contributors who submitted PRs or reported issues that were merged/closed for the 0.3.1 release:

ddbourgin
jhamrick
whositwhatnow

0.3.0¶

Version 0.3.0 of nbgrader introduces several significant changes. Most notably, this includes:

Windows support
Support for Python 3.5
Support for Jupyter Notebook 4.2
Allow assignments and students to be specified in nbgrader_config.py
Addition of the “nbgrader quickstart” command
Addition of the “nbgrader extension uninstall” command
Create a nbgrader conda recipe
Add an entrypoint for late penalty plugins

The full list of merged PRs is:

PR #521: Update to most recent version of invoke
PR #512: Late penalty plugin
PR #510: Fix failing windows tests
PR #508: Run notebook/formgrader/jupyterhub on random ports during tests
PR #507: Add a FAQ
PR #506: Produce a warning if no coverage files are produced
PR #505: Use .utcnow() rather than .now()
PR #498: Add a section on autograding wisdom
PR #495: Raise an error on iopub timeout
PR #494: Write documentation on creating releases
PR #493: Update nbgrader to be compatible with notebook version 4.2
PR #492: Remove generate_hubapi_token from docs
PR #490: Temporarily pin to notebook 4.1
PR #489: Make sure next/prev buttons use correct base_url
PR #486: Add new words to wordlist
PR #485: Update README gif links after docs move into nbgrader
PR #477: Create a conda recipe
PR #473: More helpful default comment box message
PR #470: Fix broken links
PR #467: unpin jupyter-client
PR #466: Create nbgrader quickstart command
PR #465: Confirm no SSL when running jupyterhub
PR #464: Speed up tests
PR #461: Add more prominent links to demo
PR #460: Test that other kernels work with nbgrader
PR #458: Add summary and links to resources in docs
PR #457: Update formgrader options to not conflict with the notebook
PR #455: More docs
PR #454: Simplify directory and notebook names
PR #453: Merge user guide into a few files
PR #452: Improve docs reliability
PR #451: Execute documentation notebooks manually
PR #449: Allow –assignment flag to be used with transfer apps
PR #448: Add –no-execute flag to autogradeapp.py
PR #447: Remove option to generate the hubapi token
PR #446: Make sure perms are set correctly by nbgrader submit
PR #445: Skip failures and log to file
PR #444: Fix setup.py
PR #443: Specify assignments and students in the config file
PR #442: Fix build errors
PR #430: Reintroduce flit-less setup.py
PR #425: Enable 3.5 on travis.
PR #421: Fix Contributor Guide link
PR #414: Restructure user guide TOC and doc flow to support new users
PR #413: Windows support
PR #411: Add tests for https
PR #409: Make a friendlier development install
PR #408: Fix formgrader to use course directory
PR #407: Add –no-metadata option to nbgrader assign
PR #405: nbgrader release typo
PR #402: Create a Contributor Guide in docs
PR #397: Port formgrader to tornado
PR #395: Specify root course directory
PR #387: Use sys.executable to run suprocesses
PR #386: Use relative imports
PR #384: Rename the html directory to formgrader
PR #381: Access notebook server of formgrader user

Thanks to the following contributors who submitted PRs or reported issues that were merged/closed for the 0.3.0 release:

alchemyst
Carreau
ellisonbg
ischurov
jdfreder
jhamrick
jklymak
joschu
lgpage
mandli
mikebolt
minrk
olgabot
sansary
svurens
vinaykola
willingc

0.2.x¶

0.2.2¶

Adds some improvements to the documentation and fixes a few small bugs:

Add requests as a dependency
Fix a bug where the “Create Assignment” extension was not rendering correctly in Safari
Fix a bug in the “Assignment List” extension when assignment names had periods in them
Fix integration with JupyterHub when SSL is enabled
Fix a bug with computing checksums of cells that contain UTF-8 characters under Python 2

0.2.1¶

Fixes a few small bugs in v0.2.0:

Make sure checksums can be computed from cells containing unicode characters
Fixes a bug where nbgrader autograde would crash if there were any cells with blank grade ids that weren’t actually marked as nbgrader cells (e.g. weren’t tests or read-only or answers)
Fix a few bugs that prevented postgres from being used as the database for nbgrader

0.2.0¶

Version 0.2.0 of nbgrader primarily adds support for version 4.0 of the Jupyter notebook and associated project after The Big Split. The full list of major changes are:

Jupyter notebook 4.0 support
Make it possible to run the formgrader inside a Docker container
Make course_id a requirement in the transfer apps (list, release, fetch, submit, collect)
Add a new assignment list extension which allows students to list, fetch, validate, and submit assignments from the notebook dashboard interface
Auto-resize text boxes when giving feedback in the formgrader
Deprecate the BasicConfig and NbGraderConfig classes in favor of a NbGrader class

Thanks to the following contributors who submitted PRs or reported issues that were merged/closed for the 0.2.0 release:

alope107
Carreau
ellisonbg
jhamrick
svurens

0.1.0¶

I’m happy to announce that the first version of nbgrader has (finally) been released! nbgrader is a tool that I’ve been working on for a little over a year now which provides a suite of tools for creating, releasing, and grading assignments in the Jupyter notebook. So far, nbgrader has been used to grade assignments for the class I ran in the spring, as well as two classes that Brian Granger has taught.

If you have any questions, comments, suggestions, etc., please do open an issue on the bugtracker. This is still a very new tool, so I am sure there is a lot that can be improved upon!

Thanks so much to all of the people who have contributed to this release by reporting issues and/or submitting PRs:

alope107
Carreau
ellachao
ellisonbg
ivanslapnicar
jdfreder
jhamrick
jonathanmorgan
lphk92
redSlug
smeylan
suchow
svurens
tasilb
willingc

nbgrader¶

Highlights¶

Notebook format¶

Student interface¶

Student assignment list extension for Jupyter notebooks¶

Instructor interface¶

Instructor toolbar extension for Jupyter notebooks¶

Instructor “formgrader” extension for Jupyter notebooks¶

The command line tools of nbgrader¶

The philosophy and the approach¶

How to structure course files¶

Example¶

Database of assignments¶

Configuration files¶

From the formgrader¶

From the command line¶

“Manually graded answer” cells¶

“Manually graded task” cells¶

“Manually graded task” cells with mark scheme¶

“Autograded answer” cells¶

“Autograder tests” cells¶

“Autograder tests” cells with hidden tests¶

“Read-only” cells¶

From the validate extension¶

From the command line¶

From the formgrader¶

From the command line¶

From the formgrader¶

From the command line¶

Workflow example: Instructor returning feedback to students¶

Managing the database¶

Managing assignments¶

Managing students¶

From the formgrader¶

From the command line¶

From the notebook dashboard¶

From the command line¶

From the notebook dashboard¶

From the command line¶

From the formgrader¶

From the command line¶

Directory Structure:¶

Workflow¶

Step 1: Download submission files or archives¶

Step 2: Extract, collect, and copy files¶

Autograding resources¶

Tips for writing good test cases¶

Example¶

Partially autograding, partially manually grading¶

Example¶

Checking whether a specific function has been used¶

Checking how functions were called or not called with mocking¶

Example¶

Example¶

Grading plots¶

Gotchas¶

For loops¶

Global variables in the notebook¶

Linux and Mac¶

Windows¶

API library documentation¶

High-Level API¶

Database models¶

Master version of an assignment¶

Submitted assignments¶

Gradebook¶

Installation¶

The nbgrader system and command line tools¶

nbgrader extensions¶

Installing and activating extensions¶

Installation options¶

Disabling extensions¶

Quick start¶

What is nbgrader?¶

Working with nbgrader¶

Nbgrader format¶

Course directory¶

nbgrader generate_assignment¶

nbgrader autograde¶

nbgrader generate_feedback¶

The `nbgrader_config.py` file¶

Using `nbgrader_config.py`¶