Survey Validation and Testing

Last week, I did a bit of planning regarding how best to implement the test module. I had originally been working with, but during our meeting on Friday, Presley mentioned another module called py.test which she said worked well and was simple to use. I’ll have to look into it and see which one is better for our needs. In terms of testing, I realized that I need more sample surveys that make better use of all the features (i.e. subblocks and more intricate branching). The one I have now doesn’t have any subblocking. In addition, I thought it would be a good idea to create a module full of Survey Exceptions which would be thrown if the user tries to create an invalid survey with invalid blocking and branching. Emma suggested that I add a validation check at some point before the JSON is created to make sure that all of the survey components are valid; I have determined that the best place to put this check is in the top level Survey object, in the form of a function called validate()  which is called in the JSON method. This should check for invalid blocking or branching and throw appropriate exceptions if there are issues; my test module should test that these exceptions are thrown for deliberately invalid surveys. I thought about creating checks that would throw exceptions as soon an invalid branch is created, but in order to determine whether branches are invalid, access to the entire list of blocks is needed; since this is stored in the Survey object, it is easiest to check these things at the Survey level rather than at the Question, Block, or Constraint level. This is all still in progress, and I hope to get more done on it this week. I am meeting with Presley tomorrow to go over the current Python code and discuss what needs to be done/how best to do it.

Emma and I also talked about whether it would be better to have the Python output a CSV representation of the survey rather than just spit out JSON, since the CSV is more like the “bytecode” of the survey than the JSON is (in that the JSON is just for transmitting the information, and can’t really be written by hand). CSVs are also easier to validate than the JSON. If I am to add this functionality, I’ll probably just add some methods similar to the jsonize methods to produce a CSV (I won’t get rid of the jsonize methods unless it’s clear that we don’t need them).

Regarding the branching issues from last week, I’m still not sure what the problem is. We determined that it was probably an issue with the javascript and not my JSON representation, yet when I run Emma’s HTML tests, the whole first block is skipped and the survey proceeds straight to the block question. Hopefully this is also resolved soon.

Leave a Reply

Your email address will not be published. Required fields are marked *