# Contributing to the queue (Queue) #
This guide has the following goals

1. Get a fork of the Queue repository setup. ([Getting access](#getting-access))
2. Get the project running on your machine. ([Project Setup](#project-setup))
3. Introduce the technologies used in Queue. ([Queue in context](#queue-in-context))
4. Give you some directions to navigate the codebase. ([Navigating the code base](#navigating-the-code-base))
5. Cover the process of submitting your changes! ([Submitting your changes](#submitting-your-changes))

## Getting access ##
You can fork the repository [on gitlab.com](https://gitlab.com/eip-ewi/queue).
Everybody should be able to do this provided they have a gitlab.com account.
If you are a TU Delft student or employee, please use a gitlab.com account created with your TU Delft email address.
This repository is bidirectionally mirrored, 
so any merged contributions will be automatically pushed to this repository.

After you created a fork, you can continue to the next steps.

## Project setup ##

In order to have the project setup properly for development a few steps need to be taken.
These steps assume you have already cloned the project (`git clone git@gitlab.ewi.tudelft.nl:eip/labrador/queue.git`)

1. First, the config file should be moved from the template into the actual properties file by executing
   `cp src/main/resources/application.yaml.template src/main/resources/application.yaml`
   from the root directory of the project.
2. Install [IntelliJ IDEA Professional](https://www.jetbrains.com/idea/) ([free for students](https://www.jetbrains.com/student/))
3. [Import project from Gradle model](https://www.jetbrains.com/idea/help/importing-project-from-gradle-model.html)
4. For IntelliJ to run the project make sure you are running jdk 8.
5. Start the project by right-clicking `QueueApplication` and clicking 'Run'
6. Visit http://localhost:8081/courses, please note that the regular login button currently doesn't work in the development setup,
   however visiting any other url (like /courses) in the queue will prompt you with a login screen.
   Various test users include: `Email: assistant1@tudelft.nl Username: assistant1` or `Email: student1@tudelft.nl Username: student1` and of course `Email: admin1@tudelft.nl User: admin1`. A password is **not** needed to login for the users mentioned above. 
   A list of all users and other things set up in the development environment can be found in the [DevDataBaseLoader](https://gitlab.ewi.tudelft.nl/eip/labrador/labracore/-/blob/development/src/main/java/nl/tudelft/labracore/DevDatabaseLoader.java) class in the LabraCORE repository.

## Queue in context ##
In this section we give a joint context / development view of Queue.
That is, we place Queue, the databases Queue owns, and the browser it will interact with each in its own environment.
We also identify the components Queue interacts with and provide a very high level overview of how to work with them.
More importantly this can serve as a reference to check assumptions about these tools and how we use them for the new comer, more than an introduction to start using these tools.
For that refer to section 3.

![context-view](docs/Queuecontext.png)

The diagram flows from frontend concerns on the left to back end on the right.
The queue is then made out of the following components:

1. Spring framework, with sub components:
    - Spring security
    - Thymeleaf
    - Spring repositories
2. Websockets
3. Hibernate
4. QueryDsl
5. Redis
6. Liquibase

We cover these components in two groups, the first being the more "web facing" technologies: Spring (security), Thymeleaf, Redis and Websockets.
Secondly we cover the application persistence technologies: Hibernate, QueryDsl, Spring repositories and Liquibase.

### Web technologies ###
The Queue is built using [Spring](https://spring.io/) which handles concerns such as routing, security and views.
Routing is the act of finding the right piece of code to execute for a certain http request, depending on url or the http method (think: "GET" vs "POST").
This is done by registering methods in a "controller" class and annotating them.
Spring also implements our authentication and authorisation through [Spring Security](https://spring.io/projects/spring-security).
Information about the authenticated user is stored in a session store, which, in the production environment, would be [Redis](https://redis.io).
It is not required for the development set-up, but it is an important part of the runtime environment of Queue, which is why it is listed here.

After the relevant processing is done to create the actual web page for the request, the controller method hands off control to [Thymeleaf](https://www.thymeleaf.org/).
Thymeleaf is an HTML templating engine. That is: Thymeleaf has commands that go inside HTML code that change the output HTML depending on parameters given to Thymeleaf.
These parameters are called the "model".
Think of it as the glue to only show an admin button to an admin etc.

To summarize this workflow:
An HTTP request comes in,
it is then handled by a controller method registered through Spring,
which checks if the user is allowed to do this through Spring Security,
the controller method will do some processing with our data (see the next section)
and finally create a web page for the request by handing an appropriate model to a Thymeleaf template.

A few features in the queue also have communication going to the front end that is initiated by the backend.
These include notifications and updates to the request table.
These are sent through a [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API) we don't explain anything regarding them here, but list them to note them as an important part of the runtime environment of Queue.

### Persistence technologies ###
For our persistence technologies Query Dsl, Spring repositories and Hibernate have a clear relationship with one another, while Liquibase is mostly used in isolation.
Hence we cover the first three together and cover Liquibase individually.

1. [Hibernate](http://hibernate.org/) is an ORM (object relational mapper).
  This means it knows how to connect to the database and can navigate the schema of the database by knowledge derived from classes in our code base.
  Hence, it maps objects to the relations (tables) in the database.
  You can write queries directly with hibernate as well.
  However, the level of abstraction is not much higher than that of plain sql.
  The best spot to find information about Hibernate is directly in their [user guide](https://docs.jboss.org/hibernate/orm/5.4/userguide/html_single/Hibernate_User_Guide.html).

2. [Spring Repositories](https://spring.io/projects/spring-data) are the first technology we use to raise the level of abstraction for writing queries.
  A spring repository is an interface declaration with certain method signatures describing queries.
  Spring knows from these method declarations how to generate an implementation for the intended queries through Hibernate.
  That is, Spring uses Hibernate to execute the queries.
  All these interfaces are found in the repository package of Queue.

3. [Query Dsl](http://www.querydsl.com/) is a domain specific language for writing SQL queries in java code rather than in strings.
  In order to do this, it generates classes at compile time capturing information about the schema in the database so that the types in the queries match the types of our schema.
  Like Spring repositories Query Dsl also does not talk to the database directly but does so through hibernate.

![Persistence Stack](docs/PersistenceStack.png)

Finally [Liquibase](https://www.liquibase.org/) is a tool for database migrations.
So the Queue, once it is up and running, actually has nothing to do with it.
However, it bridges changes you make to the schema in terms of the Hibernate model classes to the schema actually running in production.
The database in production is actually populated, and cannot be recreated from scratch every time the queue starts.
Liquibase is a structured way you can write down changes to the schema (that reflect the changes made in the hibernate classes) that can be applied incrementally to the production database.


## Navigating the code base ##
Aside from understanding the components included in Queue, you as a developer also need to be able to find the relevant code for each component.
Most features will affect most of the components.
That is, most features will change the view (Thymeleaf), require some logic in a controller and perhaps also store more information in the database.
We propose a way to tackle such features with the high level steps described below.

1. Find the relevant page in the running Queue (dev environment) application that you want to change
2. Find the controller method that corresponds to that page
3. Change the frontend (Thymeleaf HTML, CSS, etc.)
4. See what is required in the controller method and implement
5. Find out what changes the database needs
6. Update the hibernate model
7. Write a liquibase migration for this change
8. Hook up the controller logic
9. Edit the changelog to reflect your recent changes (take a look at [keep a changelog](https://keepachangelog.com/en/1.0.0/) for how to do this)

To find the controller method you need, you can do a global search on the (relative) url of the page you are currently on.
If you find a hit for that string in a `@RequestMapping` annotation for example:
```java
	@RequestMapping(value = "/courses", method = RequestMethod.GET)
```
That means you are probably in the right spot.
If you want to be entirely sure put a breakpoint in the method and launch the Queue in debug to see if it hits the breakpoint.
Also note that all controllers reside in the `controller` package.


To find the corresponding Thymeleaf template, look at the string that the controller method returns.
For example:
```java
		return "course/view";
```
Means that the template in `src/main/resources/templates/course/view.html` will be rendered as a reply for this http request.


The classes that make up the schema that are mapped by Hibernate reside in the `model` package.
To figure out how to properly annotate the class to make a certain schema change the [domain-model](https://docs.jboss.org/hibernate/orm/5.4/userguide/html_single/Hibernate_User_Guide.html#domain-model) section of the user guide is your best bet.


After making these changes (most likely depending on your config but not a certainty) Hibernate will crash if you restart the application.
Because the schema that is in the development database does not match the schema it expects based on the classes in the project.
To fix this mismatch we need to create a liquibase migration.
These migrations reside in `src/main/resources/migrations.yaml`.
Either check the liquibase documentation or the previous changes in the file for inspiration on how to do this.

With that in place you can update the logic in the controller to hopefully finish your new feature!

#### A small side note on configuration ####
A lot of changes to the queue also require changes to the way spring is configured in the project.
The `applications.properties` file that we had you copy over in the setup section is the most important part of spring configuration.
We do not cover the meaning of the properties here, but note it as an important file in the repository.

Besides the configuration in this file, the rest of the configuration is done in code. See `QueueApplication` for this.

## Submitting your changes ##
After building your awesome new feature time has come to submit a merge request.
The merge request template should already contain a check list for you to go through but we also list some of the steps here.

1. Make sure your code is formatted properly.
   You can check this by using two different gradle tasks: `./gradlew spotlessCheck` and `./gradlew spotlessApply`
1. Ensure your commit message has a title explaining what the change is and a body explaining why you made this change.
1. Ensure your commit message ends with a `Signed-off-by`.
   This can be automated by running a script once following [the steps below](#signing-your-work)
1. Ensure your code is tested.
1. Ensure CI passes.
1. Ensure the significant changes you made are listed in the changelog.
    1. Log changes in the [Unreleased] tag
    1. Put changes under appropriate tag, choose from list at the top

### Signing your work ###
**If you are brand new to Queue**, you'll need to run a script that adds `Signed-off-by` to each of your commits. This trailer means that the author agrees with the [Developer Certificate of Origin (DCO)](docs/developer-certificate-of-origin) and with licensing the work under the [AGPL v3](LICENSE). This is important for our team and our users, legally. The [setup_commit_msg_hook.sh](contributing/hooks/setup_commit_msg_hook.sh) script will setup a Git commit-msg hook that will add this trailer to all the commit messages you write in that repository. This is not as complicated as it sounds: it is similar to typing `git commit --signoff` every time you commit. Just download the script and run it once from the root folder of whatever Git repository you are in (for example, queue or labrador).

The Developer Certificate of Origin (DCO) is a document that certifies you own and/or have the right to contribute the work and license it appropriately. The DCO is used instead of a [CLA (Contributor License Agreement)](https://en.wikipedia.org/wiki/Contributor_License_Agreement). With the DCO, you retain copyright of your own work. The DCO originated in the Linux community, and is used by other projects like Git and Docker. To learn more about the DCO see this [posts on elinux.org](http://elinux.org/Developer_Certificate_Of_Origin) or how [docker uses the DCO](https://blog.docker.com/2014/01/docker-code-contributions-require-developer-certificate-of-origin/).

#### Commit messages ####
Your commit messages should end with the following trailer:

```
Signed-off-by: User Name <email@address>
```

where "User Name" is the author's real (legal) name and email@address is one of the author's valid email addresses.

As explained in more details above, this is easy to fix by running once the [setup_commit_msg_hook.sh](contributing/hooks/setup_commit_msg_hook.sh) script in the root folder of the repository. If you are familiar with terminals, you probably know how to do this. If you use a special application for making commits, such as Sourcetree or GitKraken, or if you are working with git bash, then please refer to the instructions below.

#### Windows / Git Bash ####
If you are using Git Bash / Git Shell / Git Gui for windows then you can do the following to setup the commit hook.

1. Open the parent directory of the repository in Explorer, and use right click on the queue directory, git bash here.
   ![Git Bash Example](docs/GitBashExample.png)
1. In the window that opens, type  
   `sh contributing/hooks/setup_commit_msg_hook.sh`  
   and press enter
1. You should see a message stating that installation is successful

#### Mac / Git ####
If you are using Git on Mac then you can do the following to setup the commit hook.

1. Open a terminal
1. Navigate to the queue folder with the cd command
1. Type  
   `sh contributing/hooks/setup_commit_msg_hook.sh`  
   and press enter
1. You should see a message stating that installation is successful

#### Sourcetree ####
If you are using sourcetree, you need to ensure that sourcetree uses the system git.

1. Install Git Bash for [Windows](https://github.com/git-for-windows/git/releases/latest) / Install git for [Mac](https://git-scm.com/download/mac)
    1. **Windows**: Open Git Bash, navigate to the queue folder and run the install script, as explained under [Windows / Git Bash](#windows___git_bash) above
    1. **Mac**: Open a terminal, navigate to the queue folder and run the install script, as explained under [Mac / Git](#mac___git) above
1. Open sourcetree and click the Settings icon in the top right corner.
1. Go to the Git tab and click the `Use System Git` button as shown in the screenshot
   ![Sourcetree Settings](docs/SourceTreeExample.png)
1. Navigate to the folder where you installed git, select the git command and press OK.
1. Your commits made in Sourcetree will automatically be signed off

#### GitKraken and Other ####
If you are using GitKraken, you can execute the following instructions. The same will also work for many other git tools. If the program you use is not listed, try follow the instructions for Git Bash (Windows) / Git (Mac) and it is likely that your commits will automatically have the sign-off message after you make them in your git application of choice.

1. Install Git Bash for [Windows](https://github.com/git-for-windows/git/releases/latest) / Install git for [Mac](https://git-scm.com/download/mac)
    1. **Windows**: Open Git Bash, navigate to the queue folder and run the install script, as explained under [Windows / Git Bash](#windows___git_bash) above
    1. **Mac**: Open a terminal, navigate to the queue folder and run the install script, as explained under [Mac / Git](#mac___git) above
1. Your commits made in GitKraken will automatically be signed off

## Production
Queue runs on queue.tudelft.nl (alias for queue.ewi.tudelft.nl).
This server is managed by the developers of the CSE Teaching Team (eip-ewi@tudelft.nl).

### Error logging
All exceptions in the production environment are logged to our [self-hosted Sentry instance](https://sentry.ewi.tudelft.nl).
Access to Sentry can be requested via the server maintainers.
In order to send the logs to the Sentry instance you need to create a properties file called `sentry.properties` next to the `application.yaml` file.
This file should contain the following (exact URL can be found in the Sentry project):

```
dsn=https://[identifier]@sentry.ewi.tudelft.nl
```

For local development your local logs should be sufficient to debug errors.
When the above key is not set in the `sentry.properties` file then the Sentry Exception Handler will be ignored.
