---
title: "Testing (or how I learned to stop worrying and love specification)"
author: David Nutter
date: May 28th 2020
output: 
  beamer_presentation:
    keep_tex: true
header-includes:
    - \usetheme[nosnake=true, nogradient=true]{BioSS}
---

# Why test

So far, we've been testing to confirm things:

- Functionality (the code does what I expect)
- Regressions (the change I made has not broken anything)
- Bugfixes (the code no longer exhibits a bug) 
  
There is another reason

# Specification

A specification defines

- What your software should do
- How it should do it (if you want to get really thorough)

. . .
Usually writing this is __BORING!__

Even more so than writing unit tests!

. . . 

And the tools are __HORRIBLE!__ 

Let me show you just how bad it can be...

# Flashback

To my undergrad years - behold:


![(Spivey, J.M. and Abrial, J.R., 1992. The Z notation. Hemel Hempstead:
Prentice Hall.)](zed-birthday-book-sample.png)



# Wossat then?

That's Z notation. 

- Clear, 
- precise, 
- provable
- next to no help when writing a program,
(or talking to a collaborator about your program). 

Two main problems:

1. It's as hard, or harder, to write a good spec as a good program (validation)
2. Checking your program conforms to the spec is hard (mapping,
   automated verification)

Also, "Fork my Z specification on github"...

. . .

...said no-one ever!

<!--- Nul points to the self-claimed "ultimate language" (Z...see what they
did there...) -->

# A closer look

![](zed-birthday-book-sample.png)

<!-- Wherein I briefly explain the spec -->

. . .

What this spec describes is slightly tricksy to do in R due to mutable state.

# But...I code by HACKING AWAY

..not writing a load of rubbish first!

And you can. You can also get a starter set of unit tests "for free"

And how? 

. . .

The answer is to write your tests *first*

Madness!?!!!

. . .

No!

This is __Test Driven Development__ (TDD) and will force you to think about two things:

1. What your code should be doing <!-- key specification question -->
2. Making your code easy to use
    * (rather than easy to write...) 

# Back to the birthday book

What should it do? Let's write a test:

```{r eval=FALSE}
BirthDayBook = ... ? ...
add_birthday = ... ? ...

test_that("addbirthday", {
	expect_equal(length(BirthDayBook), 0)
	add_birthday(BirthDayBook, "David", "1980-01-04")
	expect_equal(length(BirthDayBook), 1)
	expect_true( "David" %in% names(BirthDayBook) )
	add_birthday(BirthDayBook, "David", "1980-06-04")
	expect_equal(length(BirthDayBook), 1)
})
```

The test specifies some of the properties that any add_birthday
function must possess, just like the formal specification. 

*You can read it, if you know R*

*And we can run it, directly* 

<!-- Try __that__ with Z.  -->

# But does it pass? 

Nope! 

We have *implementation decisions* to make. But we already know:

- some of the *what* (arguments expected by add_birthday) 
- some (not all) success/fail conditions

So a good start. Further decisions:

1. *What* is BirthDayBook itself?
2. *How* should add_birthday do its thing

. . .

For 1, an environment is mutable. That'll do:

```{r eval=TRUE}
BirthDayBook=new.env()
```

# Implementing

And for 2, knowing what BirthDayBook is we can now implement the add_birthday function:

```{r eval=TRUE}
add_birthday = function(book, name, date) {
    if (! name %in% names(book)) {
        book[[name]]=as.Date(date)
    }
}
```

. . .

Later we could add more sanity checks and so on. Knock yourself out,
but a key idea in TDD is to write the simplest code possible to satisfy the
test

. . . 

Adding more functionality requires more tests

---

# Finally, verify

Then we can run the test again:

```{r eval=TRUE}
test_that("addbirthday", {
	expect_equal(length(BirthDayBook), 0)
	add_birthday(BirthDayBook, "David", "1980-01-04")
	expect_equal(length(BirthDayBook), 1)
	expect_true( "David" %in% names(BirthDayBook) )
	add_birthday(BirthDayBook, "David", "1980-06-04")
	expect_equal(length(BirthDayBook), 1)
	expect_equal(BirthDayBook[["David"]], 
		         as.Date("1980-01-04"))
})
```

And it passes!

<!-- How do I know? Well this presentation compiled didn't it? -->

(We could also do this with R.oo, closure-as-object, or in a
more R-like way in the first place (state-free))

# Things to think about when writing spec tests

Similar to when writing tests generally. How lucky!

1. "Typical" use cases (functionality check). We started with this,
   and you should too

2. Invalid input. 
   * Example: My routine expects a positive int, what should happen when I put in a
     negative float? Or NA, NULL, the contents of your sock drawer ...
   * A stop() error, warn() or some special value returned?
   * Decide *before* you write code with subtle bugs!
   
---

3. Zero, one, many. 
   * Basically what should happen when you give certain routines no
     input, a single input item or lots of input 
   * R has some "features" which make this important
   
4. Boundary conditions
   * If there's an expected change in behaviour for certain input
     values, test each side of the change point
   * Guards against off-by-one errors and suchlike
   

---

Finally, some other considerations

- concentrate your spec/testing effort on the "core" parts of your
  code that will be used heavily
- Your data input code would likely benefit from thorough
  testing
- But it's probably not worth writing loads of spec for one-off plotting
  routines and so forth.
- Though a little might help clarify your thoughts

# And when to start implementing? 

No hard and fast rules:

* When you feel you have enough spec to build a more or less correct
  implementation (of part of your software)
* When you get bored of specifying. 

Iterative development is vastly better than building a vast
software castle on shaky foundations

# Learnings

Hopefully I've shown you how test-driven development can help

- Clarify your thoughts at an early stage in development
- Avoid tedious/useless specification tasks 
- Give you a head-start on the process of software testing
- Catch a number of annoying programming errors

Obviously, I've only scratched the surface. Some resources

- The "bible" for this technique is "Test Driven Development by Example"
  (Beck, 2003). Not R-based. 
- Online _[Introduction to Test Driven Development (TDD)](http://agiledata.org/essays/tdd.html)_
- Online _[Test Driven Development in R](http://rstudio-pubs-static.s3.amazonaws.com/278724_4d8935a2955c49d9934e2113c737e70e.html)_
  
A small amount of supplementary material follows this slide

# Zero, one, many tests

R sometimes tries to be helpful. Helpful like a toddler. 

I once wrote a function that returned different column(s) from a
data.frame according to the input.

The caller always expected a data.frame though. 

Contrived example, starting with the zero case:

```{r}
foo=mtcars[,c()]
expect_true(is(foo,"data.frame"))
```

Good?

. . . 

Looks good. The "one" case:

```{r eval=FALSE}
bar=mtcars[,c("cyl")]
expect_true(is(bar,"data.frame"))
```

Problem ????

. . .

Oh dear, the type is numeric! 

----

The "many" case is OK:

```{r}
baz=mtcars[,c("cyl","disp","hp")]
expect_true(is(baz,"data.frame"))
```

. . .

So let's fix our "one" case:

```{r}
bar=mtcars[,c("cyl"), drop=FALSE]
expect_true(is(bar,"data.frame"))
```

That's better!

# Boundary conditions

Say we have some data:

```{r}
dat = sample(1:10,20, TRUE)
```

We want to replace values `1-5 with "Bad", 6-8 with "OK" and "9-10"
with "Good". We want a function called boundaries to do this:

```{r include=FALSE}
boundaries = function(x) {
	if (!is.numeric(x)) {
		stop("numeric argument required")
	}
	if (length(x) > 1) {
		return(sapply(x,boundaries))
	}
	
    if (x>0 && x<6) return("Bad")
    if (x>5 && x<9) return("OK")
    if (x>8 && x<11) return("Good") 
    stop(paste("Value ",x,"out of range")) 
}
```

```{r}
dat2 = sapply(dat,boundaries)
```

Let's test the boundaries:

```{r}
expect_equal(boundaries(1),"Bad")
expect_equal(boundaries(5),"Bad")
expect_equal(boundaries(6),"OK")
expect_equal(boundaries(8),"OK")
expect_equal(boundaries(9),"Good")
expect_equal(boundaries(10),"Good")
```

Done? 

--- 

Not quite. We also need:

```{r}
expect_error(boundaries(0))
expect_error(boundaries(11))
```

And a function:

```{r eval=FALSE}
boundaries=function(x) {
    if (x<5) return("Bad")
    if (x>5 && x<9) return("OK")
    if (x>=9) return("Good") 
    stop(paste("Value ",x,"out of range")) 
}
```

Will it pass?

. . .

Nope. Not one, but THREE bugs that will be caught by our tests.

And at least one that won't.

. . . 

```{r eval=FALSE}
boundaries(NULL)
```

Deary me. One bug per line!

# Closure-as-object implementation of BirthDayBook

Like this:

```{r}
BirthDayBook2<-function() {
    birthday<-list()
    list(add_birthday=function(name,date) {
        if (! name %in% names(birthday)) {
            birthday[[name]]<<-as.Date(date)
        }
    },
    known=function(name) {
        name %in% names(birthday)
    },
    get_birthday=function(name) {
        birthday[[name]]
    })
}
```

--- 

We use it like this:

```{r}
book = BirthDayBook2()
book$add_birthday("David","1980-10-02")

expect_equal(book$get_birthday("David"), 
	         as.Date("1980-10-02"))
```

