tech_talks/specs-without-rerun.html at master · jobteaser/tech_talks · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
<!doctype html>
<html lang="en">

  <head>
    <meta charset="utf-8">

    <title>Specs without rerun</title>

    <meta name="apple-mobile-web-app-capable" content="yes" />
    <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />

    <meta name="viewport" content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no">

    <link rel="stylesheet" href="lib/reveal.js/css/reveal.min.css">
    <link rel="stylesheet" href="lib/reveal.js/css/theme/night.css">
    <link rel="stylesheet" href="lib/template.css">

    <!-- For syntax highlighting -->
    <link rel="stylesheet" href="lib/reveal.js/lib/css/zenburn.css">

    <!-- If the query includes 'print-pdf', include the PDF print sheet -->
    <script>
      if( window.location.search.match( /print-pdf/gi ) ) {
        var link = document.createElement( 'link' );
        link.rel = 'stylesheet';
        link.type = 'text/css';
        link.href = 'css/print/pdf.css';
        document.getElementsByTagName( 'head' )[0].appendChild( link );
      }
    </script>

    <!--[if lt IE 9]>
    <script src="lib/reveal.js/lib/js/html5shiv.js"></script>
    <![endif]-->
  </head>

  <body>
    <div class="reveal">
      <div class="slides" id="content">
        <section data-markdown
                 data-separator="==="
                 data-vertical="---"><script type="text/template">

# Specs without rerun

How to write (slightly more) bullet-proof specs

===

## Your examples are not alone

---
### Your ids will change

* Even when using transactions, the ids will be incremented in the dabase
* Don't expect your ids to be the same if several other examples have occured before

```ruby
# Bad
expect(payload[:company_id]).to eq 1 # => Will work when running the file alone

# Good
expect(payload[:company_id]).to eq company.id
```

---

### Defining classes in a spec file

* Prefer anonymous classes for testing purposes
* By doing so, you will avoid any name colision

```ruby
# Bad (if the name of the class is used in another spec file)
class TestClass
  ...
end

TestClass.new

# Good
klass = Class.new do
end

instance = klass.new
```

---

### Disable caching for Rails.cache.fetch

* `Rails.cache.fetch` just ignores the configuration of the cache except for the cache store
* This has been fixed and should not happen again

```ruby
# In config/environments/test.rb
config.cache_store = [:null_store]
```
===

## Database and indexes

---
### RSpec Hooks

* The `before(:all)` and `after(:all)` hooks are not in a database transaction
* This means everything created in them will be persisted to the next spec
* You have to remove these records manually

```ruby
# Bad
before(:all) do
  @job_offer = FactoryGirl.create(:job_offer)
end

# Good
before(:all) do
  @job_offer = FactoryGirl.create(:job_offer)
end

after(:all) do
  @job_offer.destroy # Will clean the ES index
end

# Good
before(:all) do
  @job_offer = FactoryGirl.create(:job_offer)
end

after(:all) do
  after_all_cleanup # Reset all the MySQL database but doesn't touch the ES index
end
```

---

### Transactions around examples

* Another danger of using the `:all` hooks is that the object you test may be altered between two examples
* Make sure to use `ActiveRecord::Base#reload` before each example
* The transaction around the example will reset the object to its initial state
* Be on the look for memoized methods, they will not be reset when using `reload`

```ruby
before(:all) do
  @job_offer = FactoryGirl.create(:job_offer, active: true)
end

after(:all) { ... }

before(:each) { @job_offer.reload } # Without this line the examples will fail randomly

it ... do
  @job_offer.active = false
  ...
end

it ... do
  expect(@job_offer).active.to be true
end

```
---

### Don't use sleep when waiting for an index to be ready

```ruby
# Bad
JobOffer.create(...)
sleep 0.5 # Not sure that the index will be ready even after waiting for 0.5 seconds
JobOffer.search(...)

# Good
JobOffer.create(...)
JobOffer.index.refresh
JobOffer.search(...)
```

===

## Case study

---

### Symptoms

* A bunch of examples from the API controller are failing
* No error is raised, the expected results just aren't met
* The output is different for each example
* The special error handling of the API controller makes it difficult to find where the current data is different than the expected data

---

### The hunt part 1

* Method 1 - Wild guess: fail
* Method 2 - Breakpoint: fail (either it isn't reached at all or I just cannot figure what is wrong)
* At this point, it would have been a good idea to check if some examples were easier to debug than others, but having no idea they were linked together, I started with the first one
* I try to add a breakpoint earlier in the example, but it sends me through the whole Rails stack so I cannot reach our code

-

The methods I use 99% of the time will not be enough

---

### The hunt part 2

* In the middle of the stack, I have the idea to put a breakpoint in the controller to check something
* Since the controller has already been loaded, I'm forced to reload the file after adding the breakpoint
* Magic: After that, all the examples are running fine (!)

-

Reloading the code of the controller before running the example fixes the issue

---

### The hunt part 3

```ruby
Api::JobNetwork::V1::JobOffersController.new.methods.sort.each do |m|
  mapping[m] = instance.method(m).source_location.try(:join, ':')
end
```

* I run to the breakpoint again and use this snippet to find the differences in the code before and after reloading the controller
* There is indeed a difference: a callback is missing, but it refers to a dynamicaly generated method name, so no luck finding which one
* Let's search the code for any occurence of the `Api::JobNetwork::V1::JobOffersController`, nothing catches my eye
* Add a listener on every call of `require` to find in which file did the code change, but there is too much noise and I see nothing

-

I know the code has changed, but cannot find what has been changed or when it has been changed

---

### The hunt part 4

* Bring out the big guns: `Tracer`
* Start a tracer at the beginning of the example (and stop it at the end)
* Wasn't fond of this at first, because I thought it was going to be painful, but worked like a charm

```ruby
  Tracer.on
  Tracer.add_filter do |event, file, line, id, binding, klass, *rest|
    file =~ %r{/app/}
  end

  # Run the example

  Tracer.off
```

* By reading the trace, I see I'm in a place I'm not supposed to be (there is a nil variable, and a before filter is supposed to enforce a non-nil variable)
* A quick search on this before filter leads me to this:

```ruby
described_class.skip_before_filter :authenticate_api_job_offers!
```

-

Here's the culprit, the fix takes two more seconds.


---

### Lesson learned

* Using the tracer isn't that much of a hassle, and can save a whole lot of time
* Make sure to never alter the code of the application from your examples

        </script></section>

      </div>

    </div>

    <script src="lib/reveal.js/lib/js/head.min.js"></script>
    <script src="lib/reveal.js/js/reveal.min.js"></script>
    <script src="lib/jquery/dist/jquery.min.js"></script>
    <script src="config.js"></script>

  </body>
</html>