Cool article. What about hiding the test code to the students to make sure they don't write their solutions specifically to make their tests pass? Maybe using a git submodule or something?
Besides, I'd be curious to read about your app that runs the tests by itself. Are you going to open-source it? Because until now, I only found paying solutions to having a platform running my tests before I deploy, when I'm pretty sure an Heroku app could handle that pretty well...
For the first point, we had a debate on wether we should expose the tests to the student as it would:
1. Maybe help them to find the solution
2. Allow them to change the tests just to make their solution pass
In practise, we realized that error messages from specs do not necessary reveal the solution (point 1), and for point 2, we decided to trust the students (they are adults, in a paying bootcamp), and it works actually. So no need to put spec overwriting / hiding.
For the second question, we are running a simple rake runner (see https://gist.github.com/ssaunier/9713130), launching the tests in a specific folder and parsing the result. We are not doing isolated environments with ruby install / bundle install like TravisCI does, so it is suited for our simple use case only.
[+] [-] aug-riedinger|12 years ago|reply
Besides, I'd be curious to read about your app that runs the tests by itself. Are you going to open-source it? Because until now, I only found paying solutions to having a platform running my tests before I deploy, when I'm pretty sure an Heroku app could handle that pretty well...
[+] [-] ssaunier_|12 years ago|reply
For the first point, we had a debate on wether we should expose the tests to the student as it would:
1. Maybe help them to find the solution 2. Allow them to change the tests just to make their solution pass
In practise, we realized that error messages from specs do not necessary reveal the solution (point 1), and for point 2, we decided to trust the students (they are adults, in a paying bootcamp), and it works actually. So no need to put spec overwriting / hiding.
For the second question, we are running a simple rake runner (see https://gist.github.com/ssaunier/9713130), launching the tests in a specific folder and parsing the result. We are not doing isolated environments with ruby install / bundle install like TravisCI does, so it is suited for our simple use case only.
Hope that answers your question!
[+] [-] c2prods|12 years ago|reply
[+] [-] Nobigie|12 years ago|reply
[+] [-] nmayrargue|12 years ago|reply
[+] [-] paillard|12 years ago|reply
[+] [-] willybraun|12 years ago|reply
[+] [-] nicolas-lescop|12 years ago|reply
[+] [-] FabienGrenet|12 years ago|reply