Significance and Count Threshold

Why does significance change when I switch level or add waves of data in Askia Analyse?

This post explains which count is considered for the ‘Count threshold’ setting in Col sig advanced options. There is detail on how to calculate ‘Counts when independent’ and why this can change sig testing when the total base grows.


From time to time clients will raise a query regarding sig testing (Column significativity on closed) for a particular cell changing when the total base of their table grows, even though the count and % for cells in the table remains the same. E.g. this can happen when new waves of data are added or the level has been changed to show the response counts rather than respondent counts.

The reason for this is almost certainly the Count threshold setting in the col sig advanced options; it has been left as the default value, 5.

Column significativity advanced options in askiaanalyse

The Count threshold does not use the counts you would normally see in your tables but, instead, it uses the counts from the calculation Counts when independent.

You need to use the count when independent rather than the actual count because the usual count in the cell (crossed count) can still carry importance at low or zero counts.

Below is an example of when the the sig testing changes even though the count and % have remained the same (see column G)

Example of sig test change in askiaanalyse

5 is the default because this doesn’t bring to your attention differences which are not statistically important. The side effect is that you can have changes as the total base grows over time (and hence changes the count when independent). If this behaviour is not in line with requirements, the Count threshold can be changed to 0.

This applies to Column significativity on closed but you cannot perform the same calculation for Column significativity on numeric. The Count threshold setting exists in its advanced settings but it is not applicable and has no effect.

Link to this knowledge base article in our help centre

 

New Knowledge Base articles incoming!

We have been regularly updating Askia’s Help centre with various KB articles that cover our software offering. Here are some of these latest articles:

Askiadesign: about:blank error message

Microsoft recently updated Internet Explorer. This update may cause the below error message to be displayed in askiadesign:

about:blank error message in askiadesign

See the full article to fix this issue.

Askiafield: Data exports: Methods, Pros & Cons

This in-depth article covers all you need to know about the different data export methods along with the pros & cons for each method. From Supervisor’s Export to QES file to Analyse’s Data Inversion; this article also details SQL Synchronisation from AskiaTools as well as the powerful Fast Export method.

Read the full article for the detailed explanations.

 Askiaanalyse: Mean Summary Tables

This article goes through step by step examples and explanations of how to set up calculations in Analyse which will create a mean or sum summary table. There is also an introduction to script formatting and advanced sorting for such calculations.
This detailed article also provides downloadable example .QES files.

Access the KB article for more details.

Askiafield: Survey XML file is corrupt

Askiaface for iOS & Android make use of the XML version of the .QEX file; it sometimes occurs that certain characters (trailing spaces, &, >, <, …) will cause the survey’s XML definition to be invalid.

This article guides you through the necessary steps to fix a corrupt XML file: by identifying the cuplrit(s) and getting rid of them!

Check out the article for more details.

Askiafield: wildcards

This article describes how to use the Wildcard feature in Supervisor to find and update sample contacts based on regular expressions. Wildcards are a set of extremely useful tools that can power your workflow when searching / targeting specific contacts with large sample files.

Read the detailed article for more information.

Props to Seyf, Stephen & Jérome D for their contribution!