## What Are Window Functions?

At a high level window functions allow us to reason about and perform calculations on small sets of data within a larger query. They are useful for doing statistical calculations like running totals and moving averages. Initially, window functions might seem very similar to aggregate functions, but they are quite different. Window functions use values from one or many rows to return a value for each row, whereas aggregate functions return a single value for one or many rows.

Here is a quick example demonstrating the difference between window functions and aggregate functions. While the syntax of the window function may not be familiar yet, I think the explanation and query result will be helpful in understanding key differences.

This first query is a straightforward aggregate query that groups on the CustomerID column and provides an aggregate sum of TotalDue for each group of CustomerIDs.

```sql
-- Aggregate SUM() Function
SELECT SUM(TotalDue), CustomerID
FROM Sales.SalesOrderHeader
GROUP BY CustomerID
ORDER BY CustomerID;
```

![aggregate-query.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642648043/aggregate_query_a73a621c43.png)

The second query is very similar to the previous, but it includes the `ROW_NUMBER()` function that effectively numbers each row in the "window" defined by the `OVER` clause. It's key to note here that `ROW_NUMBER` is not constrained by the GROUP BY aggregation because it is evaluated afterward, so each row in the entire result set is numbered individually.

```sql
-- Window Function using ROW_NUMBER() and OVER
SELECT SUM(TotalDue), CustomerID,
    ROW_NUMBER() OVER(ORDER BY CustomerID) as RowNumber
FROM Sales.SalesOrderHeader
GROUP BY CustomerID;
```

![row_number_example.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642648043/row_number_example_46cc218a23.png)

Now that we've seen an example of a window function let's jump into the syntax used to define them in queries.

## Window Function Syntax

If you look at the documentation page for clauses involved in window functions it may seem a little overwhelming with all of the different options available. Personally, I think that the majority of uses cases for window functions can be broken down into four parts.

1. Function: performs calculations on the specified "window" of data
2. `OVER`: used to define the "window"
3. `PARTITION BY`: part of the `OVER` clause used to divide the result set into partitions
4. `ORDER BY`: an optional piece of the `OVER` clause which logically orders results in each "window"

We will look at the different kinds of window functions in the next section. First, we'll be placing our focus on the syntax and purpose of the `OVER` clause when used with `PARTITION BY` and `ORDER BY`.

The general syntax when using the `OVER` clause is relatively simple. We use `OVER` in conjunction with a function like `ROW_NUMBER()` to partition, order, and perform calculations against windows of data.

```sql
ROW_NUMBER() OVER([PARTITION BY expression] [ORDER BY expression])
```

In the example below we can use the ROW_NUMBER function to apply a number to each row in the partitions defined by `PARTITION BY CustomerId` and ordered with `ORDER BY OrderDate`. As we can see from the result set below the query, the ROW_NUMBER function is applied to each partition of CustomerIDs and it starts over with each new partition (CustomerId). The rows within each partition are ordered by the OrderDate, which is defined within the `OVER` clause and not the overall query.

```sql
SELECT SalesOrderID, OrderDate, CustomerID,
    ROW_NUMBER() OVER(PARTITION BY CustomerId ORDER BY OrderDate) AS RowNum
FROM Sales.SalesOrderHeader
ORDER BY CustomerID;
```

![row_number_partitioned.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642648043/row_number_partitioned_040866ffb5.png)

Now that we've gone over the common syntax for window functions covered let's take a look at the different types of Window Functions and what they can be used for.

## Types of Window Functions

So far we've looked at the purpose and general syntax used with window functions, but we haven't talked specifically about the different kinds of window functions and what they can be used for. In the next three sections, we'll be looking at Ranking, Aggregate, and Analytic functions which can all be applied as window functions. First up is ranking functions.

### Ranking Functions

At a high level, the ranking functions return a ranking value for each row in a result set partition. We've already seen a couple of examples of a ranking function with `ROW_NUMBER`, which returns the sequential number of a row in a partition. The other ranking functions available to use are `RANK`, `DENSE_RANK`, and `NTILE`.

The `RANK` function will rank each row in a result set partition with a numeric value. `RANK` is very similar to `ROW_NUMBER` except that `RANK` will return the same number for ties. When ranking the `RANK` function will take into account both the `PARTITION BY` and `ORDER BY` column values. If we want to rank the entire result set instead of individual partitions, we can exclude the `PARTITION BY` clause.

Run the following code to test out the `RANK` function. As you'll see from the result set rows with Id 2 and 3 have the same quantity, which leads to an equal rank. An important thing to note is that gap is left in the rankings where 3 should be leaving the ordering to be `1,2,2,4,...`.

```sql
CREATE TABLE #RankTest (Id INT PRIMARY KEY NOT NULL, Quantity INT NOT NULL);
GO
INSERT INTO #RankTest (Id, Quantity)
VALUES(1, 0),(2,1),(3,1),(4,2),(5,3),(6,5);

SELECT Id, Quantity,
    RANK() OVER(ORDER BY Quantity) AS QuantityRank
FROM #RankTest;

DROP TABLE #RankTest;
```

![quantity_rank.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642650230/quantity_rank_2d47aceab8.png)

`DENSE_RANK` is very similar to `RANK`, except that it does not leave a gap in the rankings. So if we use our previous example code, but call `DENSE_RANK`instead we'll notice that the resulting `QuantityRank` is `1,2,2,3,4...`.

```sql
CREATE TABLE #RankTest (Id INT PRIMARY KEY NOT NULL, Quantity INT NOT NULL);
GO
INSERT INTO #RankTest (Id, Quantity)
VALUES(1, 0),(2,1),(3,1),(4,2),(5,3),(6,5);

SELECT Id, Quantity,
    DENSE_RANK() OVER(ORDER BY Quantity) AS QuantityRank
FROM #RankTest;

DROP TABLE #RankTest;
```

![quantity_dense_rank.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642650230/quantity_dense_rank_0d3047ad3f.png)

`NTILE`, the last of the ranking functions to cover, is a little different in comparison to other ranking functions in that it is used to distribute rows in an ordered partition to different buckets. In the previous examples, the `ORDER BY` clause was optional, but with `NTILE` it is required in order to properly distribute rows within a partition. The best example I can think of for this function is a grading system that split results into quartiles (4 buckets). In the example below the grades will be distributed into 4 buckets (quartiles) based on the descending order of student grades.

```sql
CREATE TABLE #StudentGrades(Id int primary key not null, Grade int not null);
GO

INSERT INTO #StudentGrades(Id, Grade)
VALUES(1,88),(2,66),(3,94),(4,60),(5,100),(6,84),(7,71),(8,96),(9,73),(10,58);

SELECT Id, Grade,
    NTILE(4) OVER(ORDER BY Grade DESC) AS Quartile
FROM #StudentGrades;

DROP TABLE #StudentGrades;
```

![ntile_quartile.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642650230/ntile_quartile_b21cd4ce0b.png)

### Aggregate Functions

The aggregate functions used with the `OVER` clause are the same aggregate functions that are commonly used when grouping data with `GROUP BY`. This includes functions like `MIN`, `MAX`, `AVG` and the [rest](https://docs.microsoft.com/en-us/sql/t-sql/functions/aggregate-functions-transact-sql?view=sql-server-ver15). The only aggregate functions that can't be used as window functions are `STRING_AGG`, `GROUPING`, and `GROUPING_ID`. 

Let's continue with the student grading example by introducing a `StudentId` column and multiple test scores for each student. We will then use the aggregate functions with `OVER` to return the `MIN`, `MAX`, `AVG`, and `COUNT` for each student.

```sql
CREATE TABLE #StudentGrades(Id int primary key not null, StudentId int not null, Grade int not null);
GO

INSERT INTO #StudentGrades(Id, StudentId, Grade)
VALUES(1,1,88),(2,3,66),(3,1,94),(4,5,60),(5,16,100),(6,2,84),(7,5,71),(8,19,96),(9,16,73),(10,1,58);

SELECT DISTINCT StudentId,
    MIN(Grade) OVER(PARTITION BY StudentId ORDER BY StudentId) AS LowestGrade,
    MAX(Grade) OVER(PARTITION BY StudentId ORDER BY StudentId) AS HighestGrade,
    AVG(Grade) OVER(PARTITION BY StudentId ORDER BY StudentId) AS AverageGrade,
    COUNT(Grade) OVER(PARTITION BY StudentId ORDER BY StudentId) AS TestsTaken
FROM #StudentGrades;

DROP TABLE #StudentGrades;
```

![aggregate_window_function.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642650411/aggregate_window_function_e88e270135.png)

While this could have also been accomplished by grouping, it's good to always have other options when grouping may not be the best option for the task at hand.

### Analytic Functions

Analytic functions are the final type of window functions that we'll be covering. They are used to calculate an aggregate value based on a group of rows. The main difference between aggregate and analytic functions is that analytic functions can return multiple rows for each group whereas aggregate functions will return a single row.

I don't plan on going through all of the analytic functions because they are relatively similar in functionality and self-explanatory, but I do think it would be worth showing an example for the `LAG` [function](https://docs.microsoft.com/en-us/sql/t-sql/functions/lag-transact-sql?view=sql-server-ver15#syntax). For the full list of analytic functions, check out the documentation [here](https://docs.microsoft.com/en-us/sql/t-sql/functions/analytic-functions-transact-sql?view=sql-server-ver15).

I find `LAG` to be an interesting function (`LEAD` is a very similar) because it actually looks at the **previous** row(s) to calculate a value. When using `LAG` we must use `ORDER BY` in our `OVER` clause and it has the following parameters.

1. `scaler_expression`: value to be returned based on specified offset.

2. `offset`: the number of rows back from the current row to obtain a value.

3. `default`: value returned when the offset is beyond scope of the partition.

When using the `LAG` function the first row of a partition will use the default value because there are no previous rows. For example, we can look at multiple grades for a student and see trends in how grades change over time. You'll notice that we use the `Grade` as the default value to avoid skewing the difference column when a partition begins.

```sql
CREATE TABLE #StudentGrades(Id int primary key not null, StudentId int not null, Grade int not null);
GO

INSERT INTO #StudentGrades(Id, StudentId, Grade)
VALUES(1,1,88),(2,3,66),(3,1,94),(4,5,60),(5,16,100),(6,2,84),(7,5,71),(8,19,96),(9,16,73),(10,1,58);

SELECT StudentId, Grade,
    LAG(Grade, 1, 0) OVER(PARTITION BY StudentId ORDER BY StudentId) AS PreviousGrade,
    Grade - LAG(Grade, 1, 0) OVER(PARTITION BY StudentId ORDER BY StudentId) AS GradeDifference
FROM #StudentGrades;

DROP TABLE #StudentGrades;
```

![lag_function.png](https://res.cloudinary.com/aaron-bos/image/upload/v1642650411/lag_function_8ea2b1f2e2.png)

Window functions are a really great feature in SQL and can provide very useful functionality when used at the right time. This post covered the basics and some of the general features, but there is so much more to learn. Check out the resources below for even more information about window functions!

#### Resources

- [OVER Clause | Microsoft Docs](https://docs.microsoft.com/en-us/sql/t-sql/queries/select-over-clause-transact-sql?view=sql-server-ver15)

- [Book: Beginning T-SQL](https://link.springer.com/book/10.1007/978-1-4842-6606-9)


Being able to query and manipulate data effectively is a valuable skill for any software engineer to have in their toolbelt. In this post, we're going to take a look at Window Functions and the functionality they provide. Although this post is going to focus on using window functions with T-SQL, it's important to note that window functions are part of the ANSI SQL standard and are available in other SQL database implementations.

Learning T-SQL Window Functions

## Zsh and Oh My Zsh

Before we get too far into using the git plugin with Oh My Zsh, let's make sure we are on common ground with the basics. First, Zsh is a UNIX command-line interpreter or shell. It has a lot of added functionality compared to a standard shell, like Bash, and is great for scripting. Zsh replaced Bash as the default terminal shell on macOS in 2019. To learn more about Zsh check out the introduction [documentation](https://zsh.sourceforge.io/Intro/intro_toc.html).

Second, Oh My Zsh is an open-source framework built to make working with Zsh easier and more productive. Oh My Zsh provides a lot of great functionality out of the box but is also very customizable. Chances are if Zsh doesn't do something, then there is already a plugin for Oh My Zsh that does it. If not, Oh My Zsh was created in a way that allows us to add the functionality ourselves by adding or extending plugins. The git plugin and its functionality will be the main focus for the rest of this blog post.

Now would be a great time to install Zsh and/or Oh My Zsh if you haven't already to make sure you get the most out of the rest of this post.

- [Zsh installation](https://github.com/ohmyzsh/ohmyzsh/wiki/Installing-ZSH#install-and-set-up-zsh-as-default)
- [Oh My Zsh installation](https://zsh.sourceforge.io/Intro/intro_toc.html)

## Using the Git Plugin

Git is a very powerful command-line tool, but repeatedly typing verbose commands and options into the terminal can be time-consuming. This is where the Oh My Zsh git plugin comes in. The git plugin for Oh My Zsh provides many useful aliases and functions for common command/option combinations. Before we get a look at any functionality we need to make sure it is enabled in our Oh My Zsh configuration.

All Oh My Zsh plugins can be managed in the `.zshrc` file, which is typically located in the `$HOME` directory. We can open and view the `.zshrc` file with the following command.

```
vi ~/.zshrc
```

With the file open, we'll be looking for a line that begins with `plugins=`, which is the configuration key responsible for telling Oh My Zsh what plugins should be enabled. For example, the value of the plugins key may look something like this. You'll notice that each plugin is delimited with whitespace (spaces, tabs, newlines).

```
plugins=(git dotenv macos)
```

With the git plugin enabled we're ready to take a look at some of the aliases and functions provided.

## Git Command Aliases

At the time of writing the Oh My Zsh plugin includes 177 different aliases for git commands. There's a good chance we'll never need to learn ALL of these aliases, but I find some of the more common ones very useful for commands that I run several times a day. We'll be looking at a quick summary of the aliases that I find most handy, but you can view the full list [here](https://github.com/ohmyzsh/ohmyzsh/blob/master/plugins/git/README.md#aliases). The majority of the aliases function as you'd expect them to in terms of accepting arguments. For example, `ga .` is equivalent to running `git add .`.

- `g`: `git`
- `ga`: `git add`
- `gb`: `git branch`
- `gba`: `git branch -a`
- `gcam`: `git commit -a -m`
- `gcb`: `git checkout -b`
- `gcmsg`: `git commit -m`
- `gco`: `git checkout`
- `gd`: `git diff`
- `gf`: `git fetch`
- `gpsup`: `git push -u origin $(git_current_branch)`
- `gl`: `git pull`
- `gloga`: `git log --oneline --decorate --graph --all`
- `gm`: `git merge`
- `gp`: `git push`
- `gpf!`: `git push --force`
- `grb`: `git rebase`
- `grbm`: `git rebase $(git_main_branch)`
- `grbi`: `git rebase -i`
- `grs`: `git restore`
- `gst`: `git status`
- `gss`: `git status -s`

The list above is not exhaustive, but I think it gives a good summary of common commands. While it may take some time to change our muscle memory from the full git commands to these aliases, it's important to think about the number of keystrokes that these aliases are saving over time. If you're wondering why saving keystrokes is important, check out this [website](https://keysleft.com/) by Scott Hanselman for a quick explanation.

## Functions for Git

In the above commands, you may have noticed a couple of tokens like `current_branch` and `git_main_branch`. These are functions provided by the Oh My Zsh git plugin. While there are not near as many functions as there are aliases, they are still powerful. The full list of functions can be found [here](https://github.com/ohmyzsh/ohmyzsh/blob/master/plugins/git/README.md#functions).

The two functions listed above are pretty apparent by their name. The `current_branch` function will echo the currently checked out branch, while the `git_main_branch` will echo the `main` branch. These functions may seem basic, but they are most useful for creating our own git command aliases or scripts. I think the function that I find most useful is `grename <old> <new>`, which renames a branch from `<old>` to `<new>` on both local **and** origin remote. I have to Google this every time I need to do it before learning the function 😅.

Learning tools as developers can be overwhelming at times, but if we take the time to master the tools used day in and day out they can become a superpower instead of kryptonite.


As developers, we are exposed to many tools. How we learn and use those tools can make a big difference in our productivity and happiness. In this post, we'll be looking at the git plugin for Oh My Zsh and how it can be leveraged to increase our productivity using Git in the terminal.

Know Your Tools: Oh My Zsh and Git

## How It Started

The past two years have felt like a blur. There have been a lot of changes for me personally and throughout the world in general. At the beginning of 2021, I wanted to try something new and exciting, which is what led to this blog. I'm someone who can't stand being static. I enjoy the challenge of pushing myself and trying to grow in all aspects of my life. Lucky enough for me software development is a great career (and hobby) to scratch that itch. There is always something to learn and improve upon. Last December I was starting to feel a bit stagnant in terms of what I was learning and wanted to put myself outside of my comfort zone a bit. Coincidentally I was also reading the [Coding Career Handbook](https://www.learninpublic.org/) by [Shawn Swyx Wang](https://twitter.com/swyx), which introduced me to the concept of "learning in public". The book is full of a ton of great content, but learning in public really resonated with me for a few reasons.

1. Writing about what you're learning helps information "stick" better.
2. By learning in public you are able to help others who are in similar situations as yourself.
3. Simply putting my thoughts in public raises my standards because I want readers to feel confident that the content is accurate and up to date.

So I started this blog in January of this year (2021). I also made it a point that I wanted to create the blog myself as part of my learning journey. I set a goal to finish up the site by the end of January, which I accomplished. In fact, I published two posts in that first month which set me in motion for the rest of the year. The best piece of advice that I read prior to starting my blog was a quote from [Jeff Atwood](https://twitter.com/codinghorror), author of the [Coding Horror](https://blog.codinghorror.com/) blog and co-founder of Stack Overflow.

> When people ask me for advice on blogging, I always respond with yet another form of the same advice: pick a schedule you can live with, and stick to it. Until you do that, none of the other advice I could give you will matter.

*Picking a schedule you can live with* is exactly what I did. My schedule for 2021 was to blog twice a month. Typically I would publish around the 15th and end of each month. I was able to stick with that cadence, which gave me the confidence to keep moving forward. This blog started as a small project with unclear expectations, which led to a lot of learning over the year.

## What I've Learned

Writing blog posts is a process. Everyone's process is probably unique, but none the less there is still a process. My process starts with coming up with ideas for blog posts. Before starting this blog I didn't think I would have any trouble coming up with topics for posts, but I was wrong. After publishing my first few posts, I felt a little rushed to come up with content for the subsequent posts. Early on I noticed that topics that came up at the last minute resulted in posts that felt lower quality or rushed. This is when I learned to **start a document to track post topic ideas**. The way I use this document is by adding to it the moment I think of a blog post idea. This is helpful because it means I won't forget, but also because it gives me some time to think about whether or not that topic will be something worth writing about. Having a healthy backlog of ideas to pull from when needed goes a long way in creating content that doesn't feel rushed.

Another thing that I've learned over the course of this year is that **writing about a topic that has purpose** is much easier than a topic that is just interesting. For example, two of my most popular posts this year were [Querying JSON Data in PostgreSQL](https://aaronbos.dev/posts/query-postgresql-json) and [Updating JSON Data in PostgreSQL](https://aaronbos.dev/posts/update-json-postgresql). Both of these posts were written because I was working on a project at work that involved moving around a lot of JSON data in a Postgres database. While working on the project I noticed that (1) there wasn't a lot of quality information about working with JSON in PostgreSQL outside of the documentation and (2) I wanted to understand the core concepts better than I did at the time. This gave me the incentive to dive into the documentation and learn how to efficiently manage JSON data in PostgreSQL, which I believe led to higher quality blog posts.

The final learning that I'd like to highlight is a realization that I had after publishing a handful of posts. I learned that **writing the blog posts for myself** and not for a particular audience is key to sustaining success. Before publishing my first few posts I was so nervous to click the 'Publish' button. I was worried that someone who is smarter and more knowledgeable than me may read my post and humiliate me for it. After a while, I realized how detrimental this was to my confidence and my eagerness to write blog posts. If I spend the whole time writing a post worrying what others will think of me, then I honestly lose focus on why started doing this in the first place..**To Learn.** If I publish a post that someone doesn't agree with, then I hope they can explain or teach me a better way. In which case I will be happy to learn from them. Ultimately I blog because I enjoy the process of learning and potentially being able to help someone who comes across my posts looking to learn or solve the same problem that I encountered.

## Looking Ahead to 2022

Looking back at my blog in 2021 I'm happy with how it progressed and what I learned. It's gratifying to look at my site analytics and see my posts being viewed a couple of hundred times per day (most views coming from a handful of the same posts). Even though this year went well, I still have some ideas for what I'd like to improve upon in 2022.

### Increase Post Frequency

At the beginning of 2021, I was a little hesitant about setting my cadence for publishing posts. I didn't want to be too ambitious setting myself up for failure, but I also didn't want to be too conservative and become bored by the idea of writing posts. I think settling on two posts per month for the first year was a solid target and one could (and did) achieve. With that being said, I'd like to increase my blog post cadence goal to **three posts per month**. It's a 33% increase in published posts, which will require a bit more effort and preparation on my part. Overall I think this goal will help me become a better writer while also allowing me to potentially reach a wider audience by way of more blog posts.

### Incorporate Wider Range of Topics

Most of my blog posts in 2021 were focused on C#, SQL, or general development topics. Next year I'd like to expand the scope of topics to include a few more of my interests. Being a backend-focused engineer my posts will probably always lean towards backend-focused technology, but I would like to sprinkle in a few more posts about frontend technology, books that I've read, or general (engineering-focused) problem-solving. I don't want every post to be on a different topic, but I'd like to implement a few series of posts that focus on a particular technology or problem. That way I can organize my learning and also make it easier for readers to keep track of what they might be interested in.

### Site Improvements

One of the things that I am most proud of is my blog site itself. The site is statically generated using NextJS and hosted on Vercel. I decided to design and style the site myself using TailwindCSS. The content is hosted on Strapi (lives on a Heroku dyno using a Postgres database) and the media is hosted on Cloudinary. There are quite a few moving pieces, which can require some maintenance from time to time. I resisted the urge of making tweaks to the site itself in favor of focusing on writing blog content. I have seen posts on Reddit or Twitter from developers falling into the trap of overhauling their blog site instead of actually publishing content. This was a trap that I did my best to avoid. With that being said there are some aesthetic improvements I'd like to make in 2022.

1. Dark mode theme toggle. This is purely aesthetic, but I love dark mode in all my apps. (Q1)
2. Ability to share a blog post on Twitter directly from a post. (Q2)
3. React to posts with emojis. (Q3)
4. Search posts based on tag, title, content. (Q4)

As you can see I've marked each feature with the quarter that I'd like to implement them in. This may not seem like a lot to accomplish in a full year, but as I mentioned before I'd like to keep my focus on the publishing posts over making changes to my blog site.

The past couple of years have been challenging for everyone in many different ways. I'm thankful that I've been able to spend time with those who mean the most to me, while also pursuing my many passions. While I've highlighted a lot of great things from 2021, I'd like to call out that it wasn't always easy. Sometimes we need to take a step back and take some time for ourselves to relieve stress or pressure.

If you've read this post or **any** of my posts for that matter, thank you. Like I mentioned earlier, I mostly blog for myself but I'd be lying if I said that nice comments here and there didn't help me kick on when things got tough. I'm looking forward to the excitement and challenges that the future has in store. See y'all next year!

In January 2021 I created this blog without any expectations of what would come of it. I had a plan but wasn't sure I'd be able to execute it. In this post, I'll take a moment to reflect and look back at this year going over what I've learned, what went well, and what didn't go so well, as well as take a look ahead at next year to see how I can improve.

What I've Learned in a Year of Blogging

## The Analogy

If you've landed on this post you may be a little curious what pulling on a loose thread from a piece of clothing and writing software have to do with each other. At a high level, the activities have nothing to do with each other, but if we take the time to look a little closer I think there are some similarities to learn from.

First, let's talk about pulling on a loose thread. The thread may be on a pair of socks, a blanket, a t-shirt, or any piece of material that was sewn together. If you've never pulled a loose thread before, then the thought of doing so may not seem like a big deal. I can assure you that pulling a loose thread *can* be a big deal. The reason is that in some stitching methods pulling a single thread can cause the entire piece of clothing to unravel. There is a possibility that the loose thread will just come right out and cause no harm, but that can't be known unless we keep pulling.

Now you may be wondering, "*What does this have to do with writing code?*", so I'll do my best to explain. If we think of a codebase as an article of clothing and a bug or feature as the loose strand, we can start to see some similarities between the uncertainty of pulling a loose thread and the uncertainty of fixing a tricky bug or adding a new feature. Let's take fixing a tricky bug as an example.

When tasked with tracking down and fixing a bug in our code, the first step will be reproducing the bug. This is similar to spotting a loose thread in a pair of wool socks. There is some imperfection that is causing issues for our users. Now we need to decide how to fix the bug. In our code that may mean fixing a syntax error, adjusting some boolean condition, refactoring the code, or rewriting an entire layer of the application. With the wool sock we may want to leave the loose thread, pull it a little bit and trim it, just trim it, or try and pull the entire thread out. At this point, we are just trying to determine the best way to solve our problem.
,
Once we've decided on a course of action we dive in. After a bit of time we realize that our proposed solution may not have been as easy as initially thought. The bug permeates through several layers of our codebase and the changes begin to ripple throughout. We had chosen to pull the loose thread and we soon realized that the end of the thread may not be in sight. At this point, we have a decision to make as a software engineer and as someone who doesn't want a loose thread in our sock. We can either keep pulling the thread (expand the scope of change and potential blast radius of fixing the bug) or we can cut the thread (determine the scope of the problem and potentially plan out finely scoped changes for the future). From this point on we're going to look at these options purely from a software engineer's lense to avoid belaboring the analogy.

## Reasons to Keep Pulling

The idea of continuing to pull on the thread relates to the feeling of uncertainty and potential fear when the scope of a code change begins to expand rapidly. Personally, there have been times when I've begun making code changes with the intention of fixing a small bug, but a few hours later I find myself having to dig deeper than expected and changing a large number of files. As software engineers, we need to be mindful of this situation and make a conscious decision on how to proceed. In this section, we'll discuss a couple of reasons to continue expanding the scope of the code change to solve the issue at hand. The first reason we're going to discuss is the need to refactor.

Refactoring is the process of changing code (hopefully for the better) usually with the intention of keeping the code's behavior consistent with the previous implementation. For example, we may refactor some code to make it more readable by extracting blocks of code into separate methods or components. To come back to the thread analogy, refactoring is similar to pulling on a thread where we are confident there is an end in sight. When taking on refactoring, it's important to have a plan in place so that the code is left in a better place than when we started working on it. If we can't be confident in the benefits of a refactor, it may not be worth pulling on the refactor thread at that time.

A second reason for pulling on a particular thread is when the issue has been recurring or frequent in nature. I think this reason is most applicable to bugs or sticky parts in a codebase. For example, if a bug is reported and this is the first time we've seen the bug it may be best to find the quickest and easiest way to fix that bug. There is no point in spending a large amount of development time on a bug that occurs infrequently. On the other hand, if the reported bug impacts a large number of users or has been reported often, it may be time to pull on the thread and see what comes loose. Often times the longer a bug fix is put off, the harder it is to address. As engineers, we will encounter these decisions on a daily basis and over time the choice of a quick fix vs. expansive fix will become easier to make. Oftentimes these decisions come with trade-offs and as difficult as it may sound, we need to be **comfortable with uncertainty**.
onto
The important thing to note with continuing to pull on the thread is that it is very easy to pull too much to the point of unraveling. If we have chosen the path of pulling the thread, we need to keep the end of the thread in sight so that we can avoid unraveling the whole thing.

## Reasons to Cut the Thread

We've just discussed some reasons to keep pulling on the thread by embracing the uncertainty that can come with poking and prodding code to make it function as expected. Now we're going to go through reasons to avoid the expansive changes and cut the thread before the whole solution unravels. In a lot of ways cutting the thread may seem like the easy way out. We may opt for the quick fix, which introduces some tech debt. We may quickly bolt a new feature onto the system to avoid refactoring properly. These may sound like poor options, but sometimes they're the right option for the problem at hand.

For every reason provided to keep pulling on the thread, we have the opposite option of cutting the thread. It's possible that the current project is on a tight deadline with no wiggle room for completion or there is a bug that is affecting a large number of users. It is important for us and our teams to fit our decisions into the problem space. Although refactoring may sound exciting, it may be best to bolt on the quick feature and move on. These decisions are never easy and they're also not usually made alone. In many cases, software development is a **team sport** that requires effective teamwork and communication. We need to learn to communicate options to both technical and non-technical stakeholders to allow the most effective decisions to be made.

Having been involved in software for several years, I've come to understand that these problems are presented to us daily. I am still learning to make the right decisions, but being cognitive of the dichotomy of pulling on a thread vs. cutting it can go a long way in helping teams effectively solve the right problem at the right time.

As software engineers, we frequently tackle complex problems in code. The source of a problem's complexity may be from the problem itself, from the code's architecture, or from something else entirely. In this post, we're going to compare writing code to pulling on a loose thread from a piece of clothing. There comes a time when writing software that we need to know when to "cut the thread" or we'll risk the whole thing unraveling.

Learning When to Cut the Thread

## Null Basics in C#

Before we get too deep into handling null in our code, let's touch on the basics of null in C#. First, we use the `null` keyword to represent a null reference (a reference that doesn't refer to any object). A bit later in this post, we'll touch on how null comes into play with different C# types.

```
string nullRef = null;
```

One of the main reasons why null can be difficult to work with is the `NullReferenceException`. This exception is thrown when our code tries to access a member of an object whose value is `null`. Often times null reference exceptions will occur unexpectedly and *can* indicate a bug in our code. For example, if we expand on our previous code snippet we'll see a `System.NullReferenceException` when trying to access the `Length` property on `nullRef`.

```
string nullRef = null;
Console.WriteLine(nullRef.Length);
```

More often than not null reference exceptions will occur when a method returns null or an object is never instantiated. Null reference exceptions can be very frustrating when they occur unexpectedly, luckily C# has a lot of functionality built into the language that can help us steer clear of these null reference scenarios. The first language feature that we're going to look at is "Nullable Reference Types".

## Nullable Reference Types

"[Nullable Reference Types](https://docs.microsoft.com/en-us/dotnet/csharp/nullable-references)" is a feature that was added to C# in version 8.0, but has not been enabled in the language by default *until the release of .NET 6*. If using a version of .NET prior to .NET 6 (and still using a language version >= 8.0), nullable reference types will have to be enabled explicitly. The goal of nullable reference types is to minimize the occurrence of null reference exceptions. Let's take a closer look at this feature to see how we can leverage it in our code.

As mentioned previously, nullable reference types are not enabled by default in versions of .NET before .NET 6. There are a few ways to enable the feature which can impact different scopes of our applications. The first option to enable nullable reference types is in the `.csproj` file and impacts the entire project. Including the line below in the project file will enable nullable reference types for the entire project.

```
<Nullable>enable</Nullable>
```

If enabling nullable reference types for the entire project is not possible or ideal, then we can enable the feature using "directives" that are scoped to specific areas of the code. The directive to use is `#nullable` combined with an option like `enable`, `disable`, etc. For more information about the use of directives to control nullable reference types, check out the docs [here](https://docs.microsoft.com/en-us/dotnet/csharp/nullable-references#nullable-contexts). Now that we know how to enable nullable reference types in our code, let's see how they work and what they can do for us.

Before C# 8.0 and nullable reference types, all reference types were *nullable*. The introduction of nullable reference types provides some additional compile-time code analysis along with annotations to explicitly indicate the purpose behind our code. While using nullable reference types we are provided with null-state analysis, which analyzes our code to indicate whether an object reference may or may not be null. The null-state analysis emits warnings to indicate that dereferencing an object (aka accessing an object member) may result in a `NullReferenceException`. The null-state analysis is able to determine that an object reference **is not** null OR **maybe** null based on our code. When a variable is in a **maybe null** state, then the compiler will emit a warning to indicate that we should check for null before dereferencing the object.

A couple of examples of the warnings we may see after enabling nullable reference types are:

```
Possible null reference argument for parameter <parameterName>
Cannot convert null literal to non-nullable reference type.
```

So far we discussed how the compiler analyzes our code to determine whether or not a `NullReferenceException` is possible based on null-state analysis. Let's take a look at the variable annotations that we can use to indicate to the compiler that a value **can** be null and mitigate those error messages.

In order to indicate that a particular variable or class member can be null a `?` is appended to the end of the type declaration. As mentioned previously, enabling nullable reference types basically means that we are assuming that any reference types in our code should **never** be null. The `?` suffix on the type declaration explicitly indicates that a reference for this object **may be null**.

```
// Examples of nullable reference types
string regularString = "I'm a regular string";
string? nullableString = "I'm a nullable reference type";
StringBuilder? nullableStingBuilder = "I'm a nullable reference type too";
```

It is important to note that appending the `?` to the reference type declarations is not the same as the `System.Nullable<T>` value types. Declaring a null reference type of `string?` is still using the same underlying type of String, whereas declaring `int?` is declared using `Nullable<int>` versus the value type of `int`. I like to think of nullable reference types as more of a feature of the compiler, while nullable value types are built into the actual runtime.

When we know a reference is not null, but the compiler still emits a warning we can use the "null-forgiving operator" `!` to override the warning. This feature should be used only when needed because when using nullable reference types, the goal is for the compiler to help us by indicating when a `NullReferenceException` is possible.

```
// Using null-forgiving operator to silence compiler warning
btn!.Click();
```

Nullable reference types are a pretty new feature in C# and may not be widely used at this point, but I really think enabling the feature can improve the quality of our code in many different ways. Next, we'll take a look at operators that will help when assigning values that may be null.

## Null-Coalescing and Variable Assignment

In this section, we'll be talking about a couple of operators that help with null checking and value assignment. First, we'll be looking at the null-coalescing operator which was initially introduced in C# 6. The null-coalescing operator is defined by using `??` characters between operands like `a ?? b`. The purpose of this operator is to return the value of the left operand if it does **not** evaluate to null. If the left side is null, then the right side of `??` will be evaluated. This operator is very useful for avoiding ternary operators that check for null or for throwing an exception when a value evaluates to null.

```
// throwing an exception
records.FirstOrDefault() ?? throw new Exception("FirstOrDefault returned null");

// null-coalescing operator
var test = x ?? default;

// ternary operator
var test = x == null ? x : default;
```

Starting with C# 8 the only constraint on the left operand of the null-coalescing operator is that it cannot be a non-nullable value type. Previously the left side of `??` was constrained to a reference type or nullable value type.

Let's take the null-coalescing a bit further by looking at the null-coalescing assignment operator `??=`, which was introduced in C# 8. This operator assigns the value of the right operand **only** if the left operand evaluates to null. Where this may be useful in our code is when we have null checks that assign a variable if it is null.

```
if (x == null)
{
  x = y;
}

// Replace above with the null-coalescing assignment
x ??= y;
```

Both of these null-coalescing features can help tidy up our code and remove explicit null conditions. We have one more feature that can be used in conjunction with null-coalescing operators that will make our lives even easier when dealing with null references.

## Null-conditional Operators

The null-coalescing operators were useful for providing a "fallback" so to speak when one side of the operator was evaluated to null. Null-conditional operators also provide some safety in the event of a null operand but will return `null` if the left side of the operator evaluates to null. There are two null-conditional operators that apply to two operations, member access and element access. The member access operation is defined by `?.` and is used when accessing object members (properties, methods, fields, etc.). The element access operation is defined by `?[]` and is used when accessing elements of a collection that support access by index.

```
records?.FirstOrDefault();
password?.Length;
records?[0].Name;
```

In the examples above if `records` or `password` evaluate to null then the entire expression will evaluate to null and operations to the right of the `?` will not be evaluated. The nice thing about the null-conditional operations is that they short-circuit. So the first access operation evaluating to null will short circuit the expression and subsequent operations will not be evaluated. The example with `records?[0].Name` showcases the short circuit behavior. If `records` is `null`, then the access of the `Name` property will not be evaluated.

I mentioned that the null-conditional operators play well with the null-coalescing operators. The null-conditional operators provide safety around accessing members or elements when null is a possibility. The null-coalescing operator can provide a fallback value if the null-conditional short circuits.

```
// the null-coalescing right operand will only evaluate if the null-conditional short circuits
var name = records?[0].Name ?? "Unknown";
```

In this post, we've discussed a handful of features provided by C# that make dealing with null a bit easier. Be sure to stay up to date with language versions and features as C# is an ever-evolving language and many of these features have been introduced within the last few releases.

### Resources

[Nullable Reference Types](https://docs.microsoft.com/en-us/dotnet/csharp/nullable-references)

[Null-coalescing Operators](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/null-coalescing-operator)

[Null-conditional Operators](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/member-access-operators#null-conditional-operators--and-)


As software developers, we are bound to encounter scenarios where our code behaves in unexpected ways. Some of those unexpected behaviors could be caused by null object references. In this post, we're going to take a look at the different language constructs that C# provides for working with null. My goal is to provide information that leads to more tidy code and fewer bugs.

Hey! 👋

My name is Aaron Bos and this is my blog.

Recent Posts