UI Testing in Xcode 7(masilotti.com)
masilotti.com
UI Testing in Xcode 7
http://masilotti.com/ui-testing-xcode-7/
5 comments
I agree; of the third party libraries KIF was the nicest to work with. The biggest issue is that it doesn't run reliably enough. When we moved our code to our CI box tests would false-negative almost 25% of the time.
I tried ticking the run loop a few different ways. But the test suite will crash and burn if you start to mess with the timing of the test host or application under test.
One issue with the XCTestExpectation note is that you can't wait for something if you don't know exactly how long it will take. Hard-coding in a timeout, essentially, can make your tests very brittle. Take, for instance, testing loading content from a local web server.
I tried ticking the run loop a few different ways. But the test suite will crash and burn if you start to mess with the timing of the test host or application under test.
One issue with the XCTestExpectation note is that you can't wait for something if you don't know exactly how long it will take. Hard-coding in a timeout, essentially, can make your tests very brittle. Take, for instance, testing loading content from a local web server.
One issue with the XCTestExpectation note is that you can't wait for something if you don't know exactly how long it will take. Hard-coding in a timeout, essentially, can make your tests very brittle. Take, for instance, testing loading content from a local web server.
Ideally your UI should be decoupled enough form your web server that you can mock it. That way you only have to wait for animation delays, and your tests don't fail if your server is down.
That is if mocks fit in your testing philosophy, of course, but I think this is a place were the advantage of this approach is quite clear. And this clearly doesn't work for end-to-end tests, but those are inherently brittle.
Ideally your UI should be decoupled enough form your web server that you can mock it. That way you only have to wait for animation delays, and your tests don't fail if your server is down.
That is if mocks fit in your testing philosophy, of course, but I think this is a place were the advantage of this approach is quite clear. And this clearly doesn't work for end-to-end tests, but those are inherently brittle.
Totally agree with you there. I try to have a few levels of testing:
- Unit - everything is mocked but the object under test - Integration - common flows are worked through in code (eg. showing populated cells from a fixture JSON file) - Feature - tapping through the app with the server mocked out (via recorded server responses) - Smoke - spinning up a dev server with known data and hitting a few endpoints by tapping around the app (to ensure the "wires" or working like API keys, headers, routes, etc.)
The second two are where I see UI Testing helping the most. My comment was mostly related to the level of testing I've identified as "Smoke".
- Unit - everything is mocked but the object under test - Integration - common flows are worked through in code (eg. showing populated cells from a fixture JSON file) - Feature - tapping through the app with the server mocked out (via recorded server responses) - Smoke - spinning up a dev server with known data and hitting a few endpoints by tapping around the app (to ensure the "wires" or working like API keys, headers, routes, etc.)
The second two are where I see UI Testing helping the most. My comment was mostly related to the level of testing I've identified as "Smoke".
You can't mock everything in Apple (using Swift) unfortunately. I've resorted to use XCExpectation for physics :-/
But the other snippet, you don't have to set a time, instead of continuously checks a condition and then moves on.
But the other snippet, you don't have to set a time, instead of continuously checks a condition and then moves on.
I'm all in with UI Testing in Xcode 7, The fact that I don't have to install anything extra on my mac, or server is a huge bonus.
Author here! Happy to answer any questions here or in the comment section of the post.
[deleted]
Looks like a Huge timesaver
Hell, KIF even forced you to make sure you had accessibility strings on stuff like icons and shapes. Looks like this continues on the same path.
Regarding "There is currently no way of waiting for elements to appear or disappear."
I haven't tried this, but maybe you could just wait with a condition? https://gist.github.com/seivan/0becfffbb1d7665cebfe
Or use the XCTestExpectation stuff that came with iOS 8. https://gist.github.com/seivan/d2dee9fcff0177cb3b93