Quota: sticking to the script

Nobody likes quota. They have the off-putting echo of a well-wishing community reluctantly leaving Apartheid behind. If researchers mention quota, it’s because you did not hit the targets. If a financial director mentions them, it’s to tell you how you went over and blew the budget. You do not like quota – and us, programmers, well, it was never our favourite part of the job.

But with askiafield 5.4, we have put that behind us and made quota sexy. We have rebuilt the quota interface and the quota distribution engine.  Upgrading an interface – although time consuming – is rarely a problem. Well, we made it look cool which was quite a bit of work.

Changing the entire quota engine is not something that one should approach lightly. We did it with extra care: we put together hundreds of unit tests (where we predict and verify the output of code) and integration tests (where a full automated run of CCA is monitored and the results analysed).

This refactoring had a few goals:

  • Simplify interface(s): quota definition and the quota monitoring could be done in the same window
  • Add functionality: multiple questions, numeric, grouping responses, remove all limitations on the size of the quota tree
  • Expose through an API: the quota can be defined and monitored from a web interface – or automated from an external system (like Platform One)
  • Clarify quota scripting

This article does not focus on the actual functionalities of the quota – they are documented here – but on the impact of scripting quota through routing.

Why script quota?

Scripts are not usually used for screen-out quotas. These are usually dealt automatically (by the dialler in CATI or by the automatic settings in quota). You want 500 males in region X – once you have them, the interview is simply terminated.

Typically you need script when you want have to take a decision about which concept(s) you want to test. You first ask which ads they have seen and you decide to randomly pick 2 of them and question about them.

Ideally you want to select the ones that are the least filled – the ones furthest away in counts or lowest compared to the target percentage. And you might have weird priorities to take into account (always test your client’s brand against another one, etc…).

The rules can be complicated but we have provided simple functions for this.

5.3: the unbearable weakness of strings

In 5.3, you had the possibility of querying the state of the quota by using IsQuotaFullFor, QuotaToDo, MaxQuotaToDo, and AvailableQuota.

It did the trick for a while but there were problems:

  • It was dependent on a string (e.g, QuotaDoTo(“Region:1; Product”)). It was easy to spell it wrong and only realise that you had misspelled a question near the end of fieldwork.
  • It assumed you knew your quota tree – if you had not nested the Product within the Region (or decided to relax the rules near the end), you would get the wrong result.
  • The returned result was only looking at one quota row at a time.
  • The target counts were not taken in account to prioritise your selection.

Quota in 5.4? Sorted!

Enter 5.4 – well 5.4.4 really. We have introduced new keywords: they are methods of questions instead of functions. In other words, you write something like Gender.AvailableQuota() instead.

  • AvailableQuota: returns an ordered list of responses for the quota which are still open. The ordering is done according to the count: the first element is the response where the highest number of interviews are to be found.
  • AvailableBalancedQuota: Same as AvailableQuota but the ordering is done by the difference between targets and observed.
  • QuotaList: Same as AvailableQuota but all responses are returned (even the one ones over quota).
  • BalancedQuotaList: Same as AvailableBalancedQuota but all responses are returned (even the ones over quota).

If you want to specify some additional information about the tree you can: its works like this: Product.AvailableQuota (Gender: 1, Region :3). This means no more spelling mistakes would get in the way as the compiler would pick on the fact that you specify an incorrect question.

Another thing: if the gender and the region are specified in the interview, you do not need to indicate them but you could get information about another region for instance.

But from now on, if you need to pick 2 products to test and regardless of the nightmare of a quota tree you may have defined, you should simply write:

Dim arrProductsToTest = Product.AvailableBalancedQuota()

Return {} + arrProductsToTest[1] +  arrProductsToTest[2]

Back compatibility – what is it good for?

You know we care about it. We really wanted it to make sure that scripted surveys would work as usual. But we wanted to ensure that the old weaknesses were gone. So all previous quota functions will work with the old string… but we also took the liberty of sorting the result for your convenience… and to check the whole quota tree in case a priority at top level interfered with one of the nested quota.

So we have back-compatibility but not quite: it’s simply better and more flexible – and when the old quota tree was failing, you will get the expected results. We hope you agree.

Quota categories

The algorithm to know if a quota target applies to a given interview is actually quite complicated but we are going to explain it as simply as we can… feel free to skip this (and trust us).

Let’s imagine we have a quota tree like:

Root TO DO
1 Male 50
2 Product A 40
3 Product B 0
4 Female 40
5 Product A 15
6 Region1 10
7 Region 2 5
8 Product B 15

 

Let’s look how we run the following Product.AvailableQuota (Gender: 2) call:

  1. We will look for the availability of the first modality (then second…) – so first we will look at Product A.
  2. We count the number of targets we need to attain: one for the question object and one for each of the questions passed as a parameter (Product.AvailableQuota (Gender: 2) would mean 2 targets, Product.AvailableQuota (Gender: 1, Region 2) would mean 3).
  3. We create a quota category where we set the Product (according to step 1) and we also set the parameters
  4. For all questions used in the quota, we look in the interview to see if we have data and we set it in the category.
  5. We are going to iterate through the tree – starting at the root
  6. When we hit a response for a question that’s defined in the quota category, we either explore the sub-tree or skip the branch. For example, for Product.AvailableQuota (Gender:2), when we arrive at row 2, we would skip the entire tree and continue at row 4
  7. We count the number of questions we have found which are part of our targets (as defined in step 1). If we are looking for product A in Product.AvailableQuota (Gender: 2) we would hit that target on row 5
  8. Once we have hit the target we add all the sub-quota rows. So for product A in Product.AvailableQuota (Gender: 2) we would select the following rows 5,6,7. All the quota rows? Not quite! If the region 1 was set in the interview, we would not add row 7
  9. Once the whole tree is scanned, if we have selected 0 rows, we remove one of the targets (like Gender or Region in the Step 2 example) and start again at Step 2
  10. We would go through all the selected rows, and we would return the To Do with the most constraining value (the maximum of the minimum To Do and the minimum of the maximum To Do). Yes you might have to re-read that last sentence.
  11. Do the next response (product B) and re-start at Step 1)

There is added complexity for groups… if a response is in a group and has no targets, we use the first parent group who has a target.

If a response does not have a target, we assume that the To Do value is 1.

That’s it folks!

Conclusion

We think that AvailableQuota and AvailableBalancedQuota should cover 99% of the scripting needs. We’d love to have your feedback on this of course. We might later introduce a quota object where you will be able to query the actual min and max target or the priority… let us know when you need that and how you think it should work!

New from the vault

If like us, you’ve been busy spending your time working (building surveys, helping clients, upgrading software, reading logs, …), you may not have had a chance to keep up with the latest tips & tricks from our Help Centre. Feel free to have a flick through our latest picks:

Design & Analyse FAQs

Your first steps with Askia’s range of software are now even smoother! The askiadesign & askiaanalyse FAQs will help cover your needs when start building your first questionnaire or analysing your first survey results.

Askia analyse FAQ screenshot

Access the Design FAQ and/or the Analyse FAQ to take a look!

How to import / export Askiavista projects

Upgrading your Askiavista server environment? Using a staging server before deploying your Askiavista projects to your live environment? Look no further, this article walks us through the simple steps of using AskiaVista Tool to import and/or export Askiavista projects from one server to another.

AskiaVista Tool screenshot

This nifty application allows AskiaVista administrators to manage the import / export of Vista project related components: Portfolios, Filters, Weightings, Profiles, Calculated variables, Chart templates, Table styles, …

Check out the full article for more details.

Web survey link encryption

As from version 5.3.5, we’ve had the ability to send survey links with URL encryption / prettyfying. Up until now, encryption could only be achieved manually and was only reserved for StartSurvey link type. We have therefore created a script which allows you to encrypt survey links by batch, based on a CSV file. With this script, you can encrypt many URLs at once.

Therefore, a standard DoExternalPanel link with some extra parameters looks like:

http://mydomain.com/WebProd/cgi-bin/askiaext.dll?Action=DoExternalPanel&SurveyName=fullurl&Broker=askia&BrokerPanelId=4&tT=ad

And the encrypted version would return a URL like:

http://mydomain.com/WebProd/~jgAqch8-_cm6F4j1VIJMvSW8l_cYjUaX

Vista API tutorial

This article is a followup to the first API tutorial, it showcases a series of bite-sized examples to highlight important features of the Vista API.

askiavista API tutorial illustration

This extensive tutorial covers all the basics for Vista API development through 11 examples that feature such concepts as: displaying a cross-tab via the getPages method, securing your dashboard with a login page, grouping multiple requests to improve load management and client-side performance, retrieving a list of questions and/or responses via the askiavista.getQuestions and the askiavista.getResponses methods, adding a dropdown menu to your dashboards, … and many more!

Using Amazon S3 for resource uploads for Askiaface on iOS/Android

Using an external service to automatically upload your survey’s media captured on your iOS and/or Android devices is a very convenient way to centralise all such assets and a great method of securing all such resources.

This article covers all the steps from setting up your Amazon S3 account, checking all S3 settings and prepping your survey’s media capture questions in order to automate the upload of all media content.

Read the article for the full picture!

Big Data with just one digit

I know some of you think I only attend conferences for the free food, the drinks and the social scene. They are right – no point in me denying. But in-between parties, I tend to heal my hang-overs in the semi-darkness of conferences.

Coming back from the ASC and ESOMAR, there are a few new tendencies in the Autumn/Winter MRX fashion. Forgotten MROC, gamification, mobile research, Big Data – that’s so last year… it’s main stream, dude.

These days the cool kids talk about Automation, Data fusion, Artificial Intelligence… and the Tinderisation of research.

Automation – if you’re an assiduous reader of this blog, you know it’s coming and fully available at a software provider near you. I am not going to ramble anymore about this for now but watch this space.

Artificial Intelligence is the next big thing in Research. It has been successfully used to post-code and (less successfully) to measure sentiment in open-ended questions and tweets. It’s also good at recognising logos and objects on pictures and films, building accurate predictive models and beating me at Go (well the latter is not news and not strictly research)… but now AI is also used to merge data. There is an inconvenient truth about convenience panels… and MR data in general. If your survey is 40 minutes long (or 20 minute on a mobile device), the resulting data will be awful: the participants are either too unusual to be trusted or they don’t care because they are not incentivised.

Although there is no evidence of the length of surveys diminishing (according to SSI), every-one agrees that it needs to happen. One way is to… well… make up data. You do not ask all the questions to every-one and you copy the data around for similar looking interviews – this is called ascription (it has been around for some time). For you stat geeks out there, it’s traditionally done using the Mahalanobis distance. The new thing is to use machine learning to infer missing data. Mike Murray and James Eldridge from Research Now had a great paper about automating the splitting of surveys in chunks from their XML definition. Annelies Verhaeghe from Insites and John Colias from Decision Analyst also presented two great papers about enriching surveys with open big data.

And finally after the uberisation of research which has seen the arrival of monkeys, gizmos, nuts & limes, the new trend is the tinderisation of research. Millenials (there were boos whenever the term was used – and that was every 47 seconds) take decisions with their index. Left means no, right means yes… and survey research should follow. It’s easy to understand, fast to answer and it’s your system 1 talking… And the index is not just for decisions… the navigation of a survey should be done through flicks of the index. Almost being a millennial myself (the NSA has the names of those who are laughing), I see the attraction… and we are soon to release something code-named Jupiter that might just turn (or keep) Askia the best software for the Generations Y and Z.

Askia is changing: new digital identity

As initiated a couple of months ago in our “Askia’s mission, vision & values” blog post in which we mention the elements we have been processing over the past year (Askia’s 20th birthday!): our team has gathered to help (re)define our core statements.

In order to help us and you our clients, partners, friends, … better feel the revival of our aspiration as a business, Askia’s visual identity has undergone the same process over the past year.

We examined, reexamined, broke, rebuilt and finally achieved in evolving our brand identity without ever losing sight of our roots. From our logo to our brand colour palette, from our application icons to our print material, … we have completely redesigned our visual language in order to better mirror who we are now and where we are heading.

We have therefore began to rollout this new identity throughout our digital presence: website, blog, help centredocumentation.

Our new website

Askia website screenshot

Our new help centre

Askia help centre screenshot

Open Ends

Askia's Open Ends blog screenshot

Documentation

Askia user documentation screenshot

Conclusion

As you can see, we have already achieved quite a lot this year! But there is still more! We’ve already completely overhauled our mobile face-to-face Android app, our dev centre will be updated in the coming weeks, … Keep an eye on us, much more will be coming out in the next months!

Askiaface for Android in a material world

Askiaface for Android has just been updated to version 3.3.0 and this major update brings some long awaited features!

Our mobile face to face app for Android devices is now finally compliant with smartphones as well as tablets:

Askiaface for Android on Nexus 6P screenshot
Askiaface for Android on a Nexus 6P

The user interface has been completely redesigned based on Google’s Material design principles in order to provide a seamless experience with the modern Android environment:

Askiaface for Android on Nexus 5X screenshot
Askiaface for Android on a Nexus 5X

Of course, the application remains optimised for high density display tablets and integrates Askia’s new identity style guide:

Askiaface for Android on Nexus 7 screenshot
Askiaface for Android on a Nexus 7

This update will allow your interviewers to access all our mobile face to face features with less hassle in a completely decluttered interface!

We have also updated geolocation capture by using Android’s latest GPS API in order to provide you with more precise geolocation could your devices be on cellular or Wi-Fi!

Download or update Askiaface for Android from the Play Store now!

Latest resources for our users

Even tough we’ve been on holidays, some of our dedicated support staff have been working hard to provide you with some great resources for your upcoming surveys: help articles, new survey controls and more!

Here’s an overview of all the new goodies you’ll be able to find on our help centre:

Universe settings

This short article covers all the basics on Universes, an often misunderstood feature of askiaanalyse. It’s indeed often ambiguously connected to another Filters / Sub-populations.

Universe settings screenshot

The article in detail how Universes do not change the counts but change the percentages in your table and details each of the associated values that can be assigned to a Universe for a given table / set of tables:

  1. All interviews
  2. Use selected responses
  3. Use question base
  4. Use answering base

Check out the full article.

Simplify your data analysis with myView

This comprehensive article describes the purpose and use of the myView feature in askiaanalyse. This feature allows users to create an alternative view of the questionnaire tree. In this view you can:

  • Re-order questions / responses
  • Hide questions / responses
  • Change overall structure / indent & create chapters
  • Change captions of questions and responses
  • Create grouped or calculated responses which are assigned to questions in the myView questionnaire tree

Setting up a myView comes in handy in cases such as:

  • Survey files with long structure e.g. when the data file has lots of loops or historic questions and has become difficult to navigate through
  • Files shared between a data processing (DP) team and researchers can contain many variables that are not needed by the researchers.
  • A data file might have sections of it allocated to different researchers e.g. country specific sections. A different .mlv (myView definition file) file can be supplied to each team to show only the profile of questions they will be dealing with.

Read the full article here.

Programming Col Sig (part 4)

Part 4 of the meticulous series of articles on on Column Significance is once again a very thorough piece on this calculation and it’s various settings.

This article is specific to the Student Test using estimator & efficiency coefficient. It demonstrates how to create a table which shows the pre-set Col Sig calculation side by side with the programmed calculation for the same test. It provides all steps as well as an example questionnaire and dataset that you can access directly in the source article.

Cross video survey control

This survey control is similar to our Video control but provides extra support for all browsers and will even fallback to Flash video for those still using legacy browsers.

Cross Video ADC 2.0 header image

You can check out the demo or go grab the survey control!

Light gallery

This brand new ADC 2.0 survey control allows you to add lightboxed image galleries within your surveys. This mobile friendly gallery is fully packed with features such as: zoom in, zoom out, full-screen, keyboard controls, …

Lightbox image gallery

Go take a look at the demo or head to the article to download it!

Target

Another brand new ADC 2.0, Target provides a playful survey control for numerical loops to your respondents that will allow them to drag and drop elements on a target to assign values to each.

Target survey control

You can play around with the demo or go check the article for more information!

Enter the automation era!

It’s not new, Market Research is doing badly.

A few years back, to improve profitability, most major MR institutes have been sub-contracting Survey Programming and Data Processing to Eastern Europe or Asia. This has not been enough. The next step to increase productivity is automation. The successful launch of Zappi Store has made every one acutely aware of this.

Zappi Store uses Millward Brown or Brainjuicer’s methodology to run very formatted studies, entirely automated at unbeatable costs. They have a survey with a few customisable parameters – say the name of the brand, the logo and a list of competitors. With that, they purchase the sample and produce a PowerPoint presentation with all the key (automated) findings.  Who needs researchers and analysts anymore? Actually you only need them once – to design the methodology.

At Askia, we have always known that automation was key to improving performance. At our last user conference, we presented what clients did to automate our system… and our system was always very easy to integrate in a larger enterprise ecosystem because not all of our clients use our full range. Some just collect data, some just analyse data with Askia. So we always conceived our software like bricks in a very heterogeneous Market Research wall.

The cement to these bricks is import and export to open standards but also to produce and document APIs: Application Programming Interface – that is entry points for your own geeks to play with our toys. And if you do not have your own geeks, don’t despair: some independent geeks have decided to integrate our APIs so you can automate Askia tasks.

With the new version of AskiaField (its very sexy name is 5.4.2), we have pushed automation to a new level. You can entirely control AskiaField from a custom made application – standalone app or even a web application. Eventually this means that the AskiaField Supervisor will be a web page. But in the meantime, you can write a piece of software that creates a survey script in XML, uploads it to AskiaField, makes it live as a web survey, creates a list from your customers database, email them, cleans the data by removing speeders with AskiaTools, analyses the data and produces a report that you can email every morning to your stakeholders or provides them with a dashboard.
Automation is the future – you are going to need a few more geeks… oh and some air freshener!

AskiaAnalyse: team me up, Scotty!

The software in the Askia range have been designed to work alongside each other.  If you know how your survey tree looks in Design, you will not be surprised by how it looks in Analyse. That’s the whole point of an integrated suite.

Analyse has mainly been designed to work with a single user, creating their weightings, calculated variables and filters in their QES file.

When we realised that a lot of our users were working on continuous surveys, we introduced Surf files: the analysis definitions were stored in the Surf files so there was no problem whenever you were adding more data.

However, there were two types of user for which Askia Analyse was not performing well:

1. People importing their data from an external source (e.g. triple-S, SAV, Dimensions etc)

They would import their data, maybe create a question tree structure in Design and create multiple questions or loops in Tools. Then they would create variables, weightings and sub-populations in Analyse and save their portfolios.

Now if there was any problem with the source data, such as, additional data or cleaning errors, the whole import would have to be done again. Of course, we provided ways to improve the speed of the process:

…but we wanted to make things quicker.

2. People within large teams

In large teams, a portfolio could be created by someone and run by someone else. It’s not always the same people who create the Surf file and use it. Again we had made sure that the portfolios can be shared. If someone created a sub-population within a portfolio, it would be re-created in Analyse but what about calculated variables, recodes and weightings?

To please all these users, we have implemented a new range of developments:

myView

Firstly, we have implemented myView: it lets users re-organise and hide variables as well as hide responses and automatically associate calculated or grouped responses to questions. It’s something we have given a lot of TLC to. Since the ‘my limited view’ definition is stored outside the QES or QEW (in an .mlv file), if your survey changes (after a re-import), the myView file can be re-used (and opened automatically).

Portfolio improvements

Secondly, we have stored a few more things in the portfolio. The weightings and even the calculated variables & recodes. This means that if you use any weighting or portfolio (as well as tab-template and sub-population), their definition will be saved in the portfolio. If you re-open that portfolio in a different QES or QEW, all these definitions will be re-created automatically! – If these already exist in the survey; the system will warn you if they are different.

Creating loops

Finally, an ambitious development scheduled for 5.3.5.5. We have decided to let people create loops in Analyse (well they are called levels, aren’t they?). This means even if you have data in Surf, you will easily be able to bring questions into loops without writing edits or transforming several files in Tools.

Askiaface for iOS 3.2.0

Today, we released askiaface for iOS 3.2.0 on the App Store. This update includes the following new features:

  • Kiosque mode for iPad
  • Schedule daily survey data sync

Kiosque mode

Askiaface iOS Kiosque mode screenshot

This new feature allows you to setup an askiaface for iOS (for iPad only) questionnaire as a Kiosque survey. A Kiosque survey enables you to provide a self-interviewing environment with the following:

  • Loop interviews: as soon as the current interview is completed, a new one is automatically generated and displayed on the tablet.
  • Secured kiosque mode: users can only exit Kiosque mode after providing the per Askiafield server password.
  • Inactive kiosque interview management: if no activity is detected on an in-progress Kiosque interview for at least x minutes, the application will abandon that interview and generate a new one. Of course, you decide on a per server basis the duration of the timeout.

Askiaface iOS Kiosque mode screenshot

Disclaimer: Kiosque mode is currently offered as a freely accessible trial. Access to this feature will be subject to specific licensing.

Schedule daily sync

To accompany the above new Kiosque mode, we’ve added the possibility to schedule daily survey data syncing.

Askiaface for iOS Scheduled Sync screenshot

This will allow you to synchronise all your survey data without having to operate each Kiosque devices. Even though this feature is only available on devices on which at least one Kiosque survey has been setup, it will sync data (interviews as well as questionnaire updates) for all available surveys on the device.

Askiaface for iOS Scheduled Sync screenshot

Access the Schedule Daily sync in the app’s sidebar and set the time of day to trigger the synchronisation. You’ll of course be able to deactivate feature when you no longer need it by simply toggling the Daily Sync switch.

Miscellaneous

As usual, we have also fixed some issues:

  • Various fixes on issues with keyboard input in the Settings view.
  • Fixed a performance issue in the app views.

This update is already available via the App Store and will soon be available on your favourite Android devices!

Our latest survey controls

In order to allow our users to further engage with their respondents, we have been relentlessly adding new survey controls to our software offering. These controls, also known as ADCs (Askia Design Controls) are freely accessible to all Askia users and allow extended customisation so they fit your needs.

Below, a selection of our latest survey controls:

Responsive Table

The Responsive Table ADC allows you to embed grid (matrix) questions in your survey without having to worry about mobile and tablet respondents. The table will expand / collapse depending on your respondent’s available screen real estate.

Responsive Table Askia Design Control demo

Check out the demo survey and resize your browser to see it in action. You can also check out the dedicated knowledge base article for the full-blown details and example questionnaire.

Video

The Video ADC allows you to easily embed an .MP4 video file in your survey with all the necessary options:

  • Auto-play video
  • Show media controls (play, pause, volume & full-screen)
  • Auto-submit question after video

Video Askia Design Control screenshot

Take a look at our demo survey for this control or go through the KB article for more information.

Audio

Much like the above ADC, the Audio survey control allows you to embed an .MP3 audio file in your survey with the following settings:

  • Auto-play audio file
  • Show media controls (play, pause & volume control)
  • Auto-submit question after audio

You can play around with our demo survey or check out the dedicated article to get started.

Open

We have always provided default text inputs for our open-ended questions but this survey control will supercharge such questions with nifty options:

  • Character count / max character count
  • Email input validation
  • Password input

Open Askia Design Control demo

Access the demo survey to see it in action and check out the knowledge base article for more in-depth information.

Image Lightbox

Displaying images in your survey is straightforward but sometimes you need your respondents to actually be able to view image details and benefit from a larger version of the image in order to answer a question. This is where this ADC will help as it will allow you to provide a larger render of your image without the hassle of laying out your survey page for a wider / higher image (especially when your respondents are accessing your survey from a mobile device).

Image Lightbox Askia Design Control demo

As usual, take our demo survey for a spin or head directly to the dedicated KB article for all the details.