Version 5.3.4.0: askiaanalyse

This post is the second in a series of two that detail the key features in version 5.3.4.0 of askiadesign & askiaanalyse.

Much development work has been done in 5.3.4.0 which bring about a whole host of great new features. A detailed list of these can be found in the version roadmap. Below are five of these features.


Table of contents:

    1. Improvement of inverted data format.
    2. Nested Edges.
    3. Sig Test Options Stored In Variable / Column.
    4. Excel Export Options.
    5. Weighting Efficiency.

 


Improvement of inverted data format

Inverted data is stored and read variable by variable rather than by respondent. This generally speeds up processes such as downloading, expor ting, importing and recalculating. It is another data format which stores data in a .dat folder rather than in just a .qes file.

Inverted data format options

The ‘Re-invert database’ option in Askia Analyse is used to invert the data and the ‘Open an inverted survey’ option is used to access this data format in Askia Tools.

In Analyse 5.3.4.0 a number of improvements have been made to this format:

  • We have decided to rename the old files and not use the number of responses as an extension – this was causing unnecessary problems when a question was recalculated or if the max number of responses was manually changed in Design. The extension of the inverted data files  is always .dat.
  • The developed questions in a loop are no longer stored individually – the grayed questions always contained the information so we thought the cost of reading the whole grayed data for that question was a small price to pay compared to store all the data twice. This significantly decreases the number of files and lets you easily change your mind about the “Develop level” option in Design.
  • We store the system data – or peri-data – such as start time, end-time, IP-address, completion, …
  • The data is compressed which can gain up to 95% diminution of file size ( versions prior to 5.3.4.0 were a simple uncompressed copy of the memory)
  • We have backwards compatibility: inverted data produced with a version prior to 5.3.4.0 is read properly in Analyse. The new inverted files are readable by Vista but not by older versions of Analyse or Tools.

In most tests we have seen 85-95% decrease of the inverted database size and no noticeable deterioration in the reading speed. For example, in the last test, I reduced the size of a .dat folder from 4.2 GB to 45.0 MB. In order to convert from the old to the new format you can use the following steps:

  1. Open the old format inverted data in Askia Analyse 5.3.4.0 version or higher.
  2. File > Export a sub-population
  3. Select the new target file > select ‘All interviews’ as the sub-population > check ‘Export as inverted database’. At this stage you can also select any calculated variables to export or that can be done after by copying / importing the variables from one file to another.

 


Nested Edges

Prior to Analyse 5.3.4.0, the variables you place in your edge would be line up side by side in your edge banner. So, for example, if I have three variables in my edge with 2, 3 and 4 categories then I would have a single edge banner of 9 categories.

From 5.3.4.0 onwards you have the option to change this so that your edges build upwards and you can have two or more variables (edge banners) layered on top of each other. The result from the example above is that you have 2 x 3 x 4 = 24 crossed categories created.

The benefit of Nested edges is that they allow you to have many crossed variables / dimensions in your tables. Previously you would be limited to three before you had to start thinking about using scripts, crossed calculated variables or sub-populations.

Without setting:

Edges in askiaanalyse

Settings for nested edges in askiaanalyse

With setting: – order of nested edges depends on order of variables in Edge

Cross-tab with 2 nested edges in askiaanalyse

You can keep layering them up e.g. if we add another variable, ‘Agreement’, into our edges:

Multiple nested edges in askiaanalyse

We get the following:

Cross-tab with 3 nested edges in askiaanalyse

It’s worth noting that you cannot have a hybrid situation where you have some edges layered and some not. You will have to manage this using your columns as well as your edges.

Example of usage

 


Sig Test Options Stored In Variable

Up to now, it was only possible to store col sig options in the tab template. Therefore, the same tab template would need to be replicated for small tweaks in the testing. The purpose of this new feature is to enable more flexibility by storing the col sig options in the column profile and by doing so reduces the number of tab templates required.

The closest relation we have to this new feature in versions prior to 5.3.4.0 is ‘Specify columns’.

Specify test columns for column significativity

The only difference with the latest option is that the Test cols test is not entered into the advanced options (above left) but instead the column profile by right clicking in the columns and selecting ‘Col-sig testing’ (below).

Specify test columns in column profile

Since these options can be stored in the columns, they will also be saved in a profile or a portfolio.

Example of usage

 


Excel Export Options

Prior to Analyse 5.3.4.0, we had the set of options (A) below. Now since 5.3.4.0 we have set of options (B) which allow more flexibility in organising your Excel tabulations.

Excel export options

The following is a description of what these additional options do:

New sheet for each portfolio chapter

When you are in the portfolio view you can right-click > Insert a chapter. This has the use of keeping large portfolios tidy by organising the tab definitions into sections. With the ‘New sheet for each portfolio chapter’ option set you can now export each of these chapter sections to its own sheet in Excel. So the below will give you 2 sheets. Sheet 1 has 16 separate tables and sheet 2 has 10 separate tables.

New sheet for Portfolio chapter in askiaanalyse

New sheet for each tab definition

A tab definition is created in a portfolio every time you press the yellow folder with green arrow icon in the view below:

New sheet for each tab definition in askiaanalyse

In the example below we have created 3 tab definitions. The shown settings will output 3 sheets in Excel. Sheet 1 has 8 separate tables, sheet 2 has 8 separate tables and sheet 3 has 10.

New sheet for each table definition in export to Excel options

New sheet for each edge

We have 5 variables in the rows and four responses in the edge. Our settings dictate that we therefore have 5 x 4 tables.

With the old settings we could select:

‘One tab per sheet’ – Yes, which would give us 20 sheets in our Excel output or

‘One tab per sheet’ – No, which would give us 1 sheet in our Excel output.

One tab for every response in edge option in askiaanalyse

Now with the following settings we can create an Excel output which has 4 sheets, one for each edge response. Each of the 4 sheets contains 5 separate tables, one for each variable in the rows.

New sheet for each edge option in askiaanalyse

It’s worth noting here if you were to also set ‘One tab per sheet’ – Yes, and rerun your tables, the output would be 20 sheets. This setting overrides the others. Furthermore, if you use the three settings in conjunction with each other, the setting which produces the greater number of tables will take priority.

 


Weighting Efficiency

The weighting efficiency is a measure of how much skewing we have had to do in order to get the weights to converge – the result is a percentage. The closer it is to 100% the less skewing.

In 5.3.4.0. onwards, when you run a weighted set of tables, a file called “Report of weights.txt” is created in the .dat folder that belongs to the .qes file you have opened in Analyse. If the .dat folder doesn’t already exist, it will be created.

An explanation of the weighting efficiency formula can be found here.

Reviewing the attached WeightingEfficiency.xlsx file should also help to understand how we arrive at the weighting efficiency figure.

This looks at the 24 interviews we have in the Data.qes and for each interview lists the weighting factors in column E. In the range L2:U36 it works out, step by step, the numbers required for eventually arriving at the weighting efficiency of % 80.05722937.

Weighting efficiency in askiaanalyse

In WeightingEfficiency.rar there is a .qes and weighted portfolio. If you open these and run the results, you should see the following in the Report of weights.txt created in the .dat folder:

Weighting efficiency report in askiaanalyse

The weighting info on the first line relates to the weighting options you have set:

-Min weight-            -Max weight-            -Number of possible iterations-         -Accuracy-

Weighting options in askiaanalyse

The maximum weight displayed in this file is different to what you’ll see in the weighting options because the base calculated by summing the weighting factors (24.82096438) is different to the unweighted base (24).

If you interview 100 people, and you set your base to be 10,000,000, a “normal” interview would have a weight of 10,000

If you set the maximum weight to 5, it means the algorithm will ensure the weight will not allocate a weight 5 times more than a normal interview. In this particular example, it means it will keep all interviews’ weights under 50,000.

Version 5.3.4.0: askiadesign

This post is the first in a series of two that detail the key features of version 5.3.4.0 of askiadesign & askiaanalyse.

In askiadesign 5.2.2, we introduced new controls into our library, the ADC (Askia Design Controls).

It gives you the possibility to select which controls you would like to use on your web screen and simply drag and drop the selected control onto the screen. 

Each one of these ADCs is shipped with many settings that will allow you to fine-tune and customise the rendering of the controls

Version 1 of the Askia Design Controls (ADC 1.0) was based on Adobe Flash.

In askiadesign 5.3.3, we created the first set of new ADC 2.0 controls which are JavaScript-based.

These controls are therefore compatible with all devices and the controls includes intelligent fallbacks so if JavaScript is not enabled the controls generate classical HTML controls. 

As with ADC 1.0 (Flash), ADC 2.0 allows you to customise the appearance as well as the interaction of your survey controls by giving you access to their design parameters.

Askia Design Controls in askiadesign

When generating the screens in HTML, we can also automatically associate an ADC to a given type of questions.

We can assign different controls to single, multiple or grid questions and predefine their properties.

Screen generation options for ADCs in askiadesign

In askiadesign 5.3.4.0, we have improved the management of question compatibility when you drag and drop an ADC to a response block.

AskiaDesign checks if a control requires a loop or not.

It also indicates the list of questions types it was designed for when it indicates the control is not compatible.

ADC compatibility in askiadesign

Askiadesign  automatically creates a response block when dragging an ADC on to incompatible screens.

Below the list of gender are as classical responses but the ADC should be applied to a response block.

Automatically generate response blocks for ADCs in askiadesign

After the drag and drop of the ADC, the list of classical responses is automatically transformed into a response block and the selected ADC was applied as well.

Askia Design Control generated in askiadesign

Have a look at the list of ADC 1.0  and ADC 2.0 controls.

The Electric Kool-Aid Askia Test

Abstract: Survey scripting and coding have lots in common and we should bring testing techniques into Survey Design. For this we have improved Random Data Generation and created a new Tools module called “Script Verification”.

SurveyMonkey, Google consumer surveys and other disruptive DIY technologies have changed the Market Research industry. Any marketing director can put together an online survey, get sample from a number of panel providers and have results to their strategic questions in hours.

But Askia software is not designed for marketing directors. It has been conceived for survey specialists, scripters, data processors who design and analyse complex surveys – sometimes long, sometimes algorithmically challenging, over long periods of time and eventually collecting millions of records. And with our target audience in mind we are continually improving our range of software. We want any design to be achievable, any layout, any number of records. It would be an exaggeration to call it Big Data but let’s say we specialise in “Medium Data”.

On the subject of interview data, I will only mention that in the last 2 years we have completely overhauled our way of storing data in SQL Server (5.3.3) and a new compressed inverted data format (5.3.4). But I am digressing, the subject of this article is about methodologically managing complexity.

Managing complexity with Askia's survey software

The challenge with complexity is that it invariably leads to human errors, their number exponentially growing with size, harder to spot and often too late. The thing is we, as programmers, know about complexity. Askia software is made up of millions of lines of code and, as some of you may have fleetingly experienced, it sometimes breaks. And, believe it or not, we coders have an aspiration to perfection: we constantly try new methodological or technical ways of testing our software so it works smoothly the second we release it. But any program that does anything more than sorting three numbers is bound to break and we have to live with the fact that we will always deliver short of what we wished for – but hopefully learn from past mistakes.

Survey scripting is programming – unfortunately Market Research tools are a little bit behind (yes we are aware of our responsibility there). Our first version of Design in 1994 was attempting to mimic the revolution Visual Basic brought to the programming world in 1991. All basic functionalities were available in a Graphical User Interface. We made the layout WYSIWYG but we still allowed programming in event driven scripts but hidden from the interface. Our AskiaScript still has the traces of its ancestry with variables defined with Dim, For Next loops – I’ll admit that not everybody at Askia thinks it’s a good thing but that’s the price to pay for backwards compatibility.

Reusability & object-oriented programming by Askia

Reusability is the key to decrease development time and increase reliability. For programmers reusability is generally known as Object-Oriented Programming. In all of our software, we have tried to include reusable objects: Generations Settings and Internet options in askiadesign, Tab-templates and clones in Askiaanalyse, survey inheritance in askiasurf, libraries everywhere. Last but certainly not least we have created Askia Design Controls: we have enabled (advanced) users to generate the perfect HTML / JavaScript for each PC / tablet / mobile target whatever the browser, its version or its Operating System. ADCs encapsulate data, they are polymorphic (you can use them on different types of questions and browsers) and because they are open source, it’s up to you to give them inheritance.

There is another part of programming that we would like to bring to the Market Research industry: it’s testing – unit testing, integration testing, system testing. For the development of AskiaScript 2.0 we designed the tests before we wrote one line of code – this is called test-driven development (TDD). The number of bugs was minimal for a development of that size. Each time we found a fault, we added it to the list of tests to make sure it would never surface again in subsequent versions.

Test-driven development in survey design by Askia

Along with the spec of a survey, there should be a list of tests. These tests should be run by someone other than the survey scripter – and the tester should not peek into the routing coding. Different people will think differently ensuring your tests cover more defects.  We have put together a non-exhaustive list of tests:

  • Interview level: data presence for mandatory questions, skip routing testing, coherence between questions, testing links and response visibility.
  • Usability testing: testing each screen on every platform.
  • Aggregated testing: making sure quotas are respected, rotations are balanced, multiple questions have multiple responses.
  • System testing: ensuring the survey runs well on the server and that the data you produce is usable.

Long before considering a soft launch the simplest way to see if your survey runs correctly is to generate random data. You have two ways of doing so: either by using AskiaTools’ random data generator or using a JavaScript simulator (see here). The JS simulator is a great way to achieve systems testing.

System testing can also be achieved by exporting test interview data as .dat files and looking at the size of the individual dat files: you will be able to measure the load that will incur on memory. Multiply this by the number of concurrent interviews you expect and you will have an idea of the specs you need on your server(s). Additionally, looking at the size of a .QES file or preferably of the tables generated in SQL server will indicate how much Hard Drive space you will need.

Random Data Generation in askiatools

We have recently added a lot of features to the Tools random data generator: you can define routings that would only be run during random generation (for validating the screening for instance), you can specify the behaviour when blocking error messages are displayed and more importantly you can import your quota settings and take them into account in your generation (all available in 5.3.3). Quota code is often complex and going over could be an expensive mistake.

We have also created a brand new module in Tools 5.3.5 called “Verification scripts” (see here for more details). This allows a tester – remember not the survey scripter – to create checks in AskiaScripts that will be run on each interview. So you can verify that the question about credit cards has been asked if the interviewee has mentioned banks in another question. You simply write a check like this:

Assert.Check( Banks <> {} and CreditCards.HasNA,” Interviewee should have been asked the question about credit cards” )

The scripts can be as long as you like, we have added If Else conditions and Goto to help you create complex code that you will keep in one single text file. And you can write it within an environment – the askia visual studio – where you get help and documentation on any objects, methods or keywords. You can run this on your randomly generated data, on your soft launch or on your full data set – each time you get a detailed report of how many checks have failed. At the time of writing, this is not released yet but contact us if you want to try a beta.

In these scripts, we also want to have access to aggregated data… this will allow us to have one script that runs interview level testing and aggregated testing. We might want to test if an interview took less than 10% of the average length or if a given response to a question is outside a percentile. In other words, you might want to compare interview responses not with other interview responses but all other interview responses. The script grammar for this will be described in a forthcoming article – we are still passionately discussing it internally.

Usability testing in survey design by Askia

We have not covered usability testing here – not that we do not think it’s important: we are constantly talking about it internally. We are putting together a range of tools for designing ADCs (so far codenamed ADCUtil – yes we need something catchier), we have added ways of visualising your HTML in other browsers in Design. But we need to understand when a display no longer works because of screen size, the bias triggered by no longer using JavaScript, count the number of heads of Internet Explorer 5 users – and there again we need your input and your ideas so we can automate these tasks.

In the meantime, I leave you with these great quotes:

The act of maintaining software necessarily degrades it.” – Alain April

It’s harder to read code than write it.Joel Spolsky

If you can’t measure it, you can’t improve it. Peter Drucker

Cumulative updates & future milestones

Over the past month, we have slightly shifted our development & release process with askiavista: instead of longer development cycles with many fixes & improvements, we have began to release smaller versions that mainly include fixes where new features are added at a slower pace.
This helps us maximise our response rate when fixing urgent issues. So, we are doing our best to release early, release often.

Below, a breakdown of the two latest minor updates for askiavista:

Version 6.0.3.1

released 21/11/2014

  • Add footnotes property to tables (new)
  • Export all in one page to Excel (fix)
  • Czech characters not rednered in MS Excel export (fix)
  • Infinite loader on Results when removing a raw data variable (fix)
  • Raw data response sorting not maintained when saved to Portfolio (fix)
  • User level survey offline status not taken into account (fix)

Version 6.0.3.1

released 24/11/2014

  • View switcher breaks when deleting Profiles and/or Portfolios (fix)
  • AskiaVista6 unable to open survey with numeric name (fix)
  • Vista6 – does not remove selected question when the question appears more than once in R/C/E (fix)
  • Calculated responses don’t compute the correct amount of responses (fix)
  • Portfolio linkID cannot be customised (fix)
  • Superpose response is applied when it shouldn’t be (fix)
  • Sub-population by GUI from V5 is not well interpreted in V6 (fix)
  • HTML encoding issues (fix)
  • Adding / removing a Profile or Variables empties the list of Profiles or Variables (fix)
  • Online – offline survey status causing issues (fix)

Future milestones

6.0.3.3
  • Export to PDF (new)
  • Highcharts not displaying Mean & combination charts (fix)
  • The chart is not saved into the template (fix)
  • Issue with fixed header in Firefox (fix)
  • Assigned factors not saved in Portfolio (fix)
  • Empty table when deactivating all calculations for tables (fix)
  • Erratic results in Search questionnaire (fix)
6.0.4.0

The integration of askiavista into askiaadmin that is planned for version 6.0.4.0.

Later

The improvement of AskiaVistaServer farming. This major overhaul has not yet been assigned to a version (and it may actually be introduced before the askiaadmin integration.

Significance and Count Threshold

Why does significance change when I switch level or add waves of data in Askia Analyse?

This post explains which count is considered for the ‘Count threshold’ setting in Col sig advanced options. There is detail on how to calculate ‘Counts when independent’ and why this can change sig testing when the total base grows.


From time to time clients will raise a query regarding sig testing (Column significativity on closed) for a particular cell changing when the total base of their table grows, even though the count and % for cells in the table remains the same. E.g. this can happen when new waves of data are added or the level has been changed to show the response counts rather than respondent counts.

The reason for this is almost certainly the Count threshold setting in the col sig advanced options; it has been left as the default value, 5.

Column significativity advanced options in askiaanalyse

The Count threshold does not use the counts you would normally see in your tables but, instead, it uses the counts from the calculation Counts when independent.

You need to use the count when independent rather than the actual count because the usual count in the cell (crossed count) can still carry importance at low or zero counts.

Below is an example of when the the sig testing changes even though the count and % have remained the same (see column G)

Example of sig test change in askiaanalyse

5 is the default because this doesn’t bring to your attention differences which are not statistically important. The side effect is that you can have changes as the total base grows over time (and hence changes the count when independent). If this behaviour is not in line with requirements, the Count threshold can be changed to 0.

This applies to Column significativity on closed but you cannot perform the same calculation for Column significativity on numeric. The Count threshold setting exists in its advanced settings but it is not applicable and has no effect.

Link to this knowledge base article in our help centre

 

Myforce opens a new office in Prague!

I have a bit of important MyForce news to share with you: we opened a new office in Prague, Czech Republic!
Kateřina Marková (whom we have worked with for many years already) becomes managing director of MyForce CZ.

As Myforce keeps on growing both in turnover and in number of countries where our solutions are installed, opening MyForce CZ was a logical next step for us. Choosing Prague as our new office location is no coincidence, as we had good reasons to opt for this particular city: First of all, this will help us to support a number of important local MyForce customers. Secondly, MyForce has always had close relationships with Kateřina Marková, as we have been working with her on local projects for many years now. And finally, we are certain that having a Czech office will aid us in sustaining our relationship with Brno’s University of Technology.

Kateřina Marková, managing director MyForce CZ, added: “Becoming part of the larger MyForce group ensures stability and business continuity for our local clients. With the resources of the whole MyForce team at our disposal, we can provide an even better service.

As a result, Myforce has updated the website as well: www.myforce.cz is available for those of you who know Czech 😉

MyForce CZ
Počernická 272/96
108 00 Praha 10, Czech republic
+420 272 731 711
info@myforce.cz

New AskiaVista AJAX API 2.2.0

The new AskiaVista AJAX API (2.2.0) is now released with the latest 6.0.3.x version of AskiaVista.

The documentation is now up-to-date

List of new features/fixes:

  • Add `footnote` parameter in cross-tab query (settings.results.footnote)
  • Add `language` parameter in queries to specify the survey language. It could also be add on the config, or specify via url
  • Add a new `languageUrlParam` in the configuration to read the survey language via the URL
  • Add methods:
    • getPortfolioLinks
    • getPortfolioLink
    • savePortfolioLink
    • deletePortfolioLink
    • getPortfolioLinkLayouts
    • getPortfolioLinkThemes
    • savePortfolioLinkTheme
    • deletePortfolioLinkTheme
    • getResources
    • saveResource
    • deleteResource
    • htmlEncode
  • Take in account the new calculations and their advanced options
  • Add the `baseUrl` key in the config. Strongly recommended to use it instead of `url`
  • Manage the new `rawdata` and `sectionpage` portfolio item type
  • Add a dedicated exception severity flag when the authenticity token in cookie is bad or obsolete, it then ask again the login form
  • Manage severity on server-side error to throw the `securityError` when it’s needed
  • Error messages are now htmlEncoded in the client side.
  • Security error don’t block the query stack, it now fire the security error and the regular error event

This askiavista is massive!

We have finally released version 6.0.3.0 of askiavista! First, of all, don’t judge a book by looking at the cover: this is more a 6.1 version than a minor version. Indeed, it packs:

  • 22 new features
  • 22 bug fixes
  • 7 miscellaneous tasks

Below, some of the key new features:

Portfolio section pages

Askiavista portfolio section pages

Users can now create Section pages within their Portfolios; these pages allow to add formatted text and media (image & video).
Such pages are a great solution for report introductions, notes, metadata, …

New portfolio link management

Askiavista new portfolio by link management

Portfolio links have been completely overhauled in order to allow many nifty features such as Expiry dates, Filter management, dynamic tabbing, custom URLs, default page, …
Portfolio links can also be duplicated in order to batch create multiple links.

Reorder portfolio items

Reorder portfolio items in askiavista

This was a recurrent client request: having the possibility to reorder elements (table definitions & now section pages) from within a portfolio.

Raw data / open-ended management

Askiavista raw data and open-ended management

Askiavista is now able to render data at interview level and can also render open-ended data! These flat tables are accessible in a dedicated view and can of course be embedded within a Portfolio.

Fixed headers

Fixed headers for large tables in askiavista

In order to improve our user experience when browsing large cross-tabs, we have added fixed headers (for Rows, Columns & Edges) to the Default table style.

Select all / unselect all in Dimensions

Select all - unselect all in askiavista's dimensions

This client request allows users to select all questions or response items in a given Dimension (R/C/E). This is especially useful when dealing with many questions or for very long lists of response items to which you may want to apply a given action (group, hide, …).

View source

View crosstab source in askiavista

This option (hidden by default), allows advanced users and API developers to embed table & chart data directly in their web page; could it be an askiavista-driven dashboard, portal or web page.

Compressed inverted data

We have decided to improve the inverted format:

  • We decided to rename the old files and not use the number of responses as an extension – this was causing unnecessary problems when a question was recalculated or if the max number of responses was manually changed in Design. The extension is always .dat.
  • The developed questions in a loop are no longer stored individually – the greyed questions always contained the information so we thought the cost of reading the whole greyed data for that question was a small price to pay compared to store all the data twice
  • We store system data -or peri-data – such as start time, end-time, IP-address, completion, …
  • The data is now compressed
  • We have backwards compatibility: inverted data produced with a version prior to 5.3.4 is read properly but the new inverted files are not readable by Vista (yet) or an old Analyse / Tools

We have seen up to 90% decrease of the inverted database size and almost no noticeable deterioration of the reading speed

For the full list of new features & bug fixes, check the changelog.