Pollster Rankings Methodology

Consistency Score methodology:

For a given pollster, ‘a‘, whereas ‘x’ represents a given political party:

For pollster ‘a‘:

Whereas:

Ca is defined as the Consistency Score for pollster ‘a‘;
Cb is defined as the Consistency Score for pollster ‘b‘; and
Cz is defined as the Consistency Score for the zth pollster
   
Sa is defined as the Average Error Score for pollster ‘a‘;
Sb is defined as the Average Error Score for pollster ‘b‘; and
Sz is defined as the Average Error Score for the zth pollster
   
Wa is defined as the Weighted Error Score for pollster ‘a‘;
Wb is defined as the Weighted Error Score for pollster ‘b‘; and
Wz is defined as the Weighted Error Score for the zth pollster
   
Ea is defined as the Relative Error Score for pollster ‘a‘;
Eb is defined as the Relative Error Score for pollster ‘b‘; and
Ez is defined as the Relative Error Score for the zth pollster
   
x is defined as a given political party. See “n” for more information.

(ex. x = 5 = Green Party or x = 1 = Conservative Party)
   
n is the total number of political parties a given pollster has evaluated by publishing a poll within two months of an election. For the purposes of these calculations, all provincial New Democratic Parties and the federal NDP are considered to be of the same political party. The Bloc Québécois and the Parti Québécois are also considered to be of the same political party. All provincial liberal parties are also considered to be of the same political party as the Liberal Party of Canada (including officially non-aligned and independent provincial parties such as the Liberal Party of Ontario and the Parti Libéral du Québec). All provincial green parties are considered to be of the same political party as the Green Party of Canada. All provincial Progressive Conservative parties, the British Columbia Conservative Party and the Saskatchewan Party are considered to be of the same political party as the Conservative Party of Canada. The Wildrose Party of Alberta is considered as its own entity and is not considered aligned with any other political party. Independents are not included in this calculation.
   
k is defined as any given election.

(ex. 2011 Ontario General Election or the 2009 Nova Scotia General Election, et cetera…)
   
h is the total number of elections a given pollster has participated in by publishing a poll within at least two months of election day.

(ex. Currently, for Angus-Reid, h = 22, for Harris/Decima, h = 17, et cetera…)
   
Rxk is defined as the election result of a given party, x, as reported by the election authority of the jurisdiction in question for a given election, k. It is expressed as a percentage of total number of valid votes cast. For the purposes of this calculation, federal election results are broken down by province or region such that any given federal election is considered to comprise separate six separate ‘Rx values, one each for Ontario, Quebec, British Columbia, Alberta, the Prairie region (Manitoba and Saskatchewan) and Atlantic Canada (Nova Scotia, PEI, New Brunswick and Newfoundland and Labrador).
   
axk is defined as the predicted election result for a given party, x, as determined by the final poll published by the polling firm ‘a‘ before a given election, k.

(ex. The predicted election result of the Manitoba New Democratic Party published in the last poll by the Angus Reid polling firm before a specific Manitoba general election.)



Average Error Score methodology:

For a given pollster, ‘a‘, whereas ‘x’ represents a given political party:

For pollster ‘a‘:

Whereas:

Ca is defined as the Consistency Score for pollster ‘a‘;
Cb is defined as the Consistency Score for pollster ‘b‘; and
Cz is defined as the Consistency Score for the zth pollster
   
Sa is defined as the Average Error Score for pollster ‘a‘;
Sb is defined as the Average Error Score for pollster ‘b‘; and
Sz is defined as the Average Error Score for the zth pollster
   
Wa is defined as the Weighted Error Score for pollster ‘a‘;
Wb is defined as the Weighted Error Score for pollster ‘b‘; and
Wz is defined as the Weighted Error Score for the zth pollster
   
Ea is defined as the Relative Error Score for pollster ‘a‘;
Eb is defined as the Relative Error Score for pollster ‘b‘; and
Ez is defined as the Relative Error Score for the zth pollster
   
x is defined as a given political party. See “n” for more information.

(ex. x = 5 = Green Party or x = 1 = Conservative Party)
   
n is the total number of political parties a given pollster has evaluated by publishing a poll within two months of an election. For the purposes of these calculations, all provincial New Democratic Parties and the federal NDP are considered to be of the same political party. The Bloc Québécois and the Parti Québécois are also considered to be of the same political party. All provincial liberal parties are also considered to be of the same political party as the Liberal Party of Canada (including officially non-aligned and independent provincial parties such as the Liberal Party of Ontario and the Parti Libéral du Québec). All provincial green parties are considered to be of the same political party as the Green Party of Canada. All provincial Progressive Conservative parties, the British Columbia Conservative Party and the Saskatchewan Party are considered to be of the same political party as the Conservative Party of Canada. The Wildrose Party of Alberta is considered as its own entity and is not considered aligned with any other political party. Independents are not included in this calculation.
   
k is defined as any given election.

(ex. 2011 Ontario General Election or the 2009 Nova Scotia General Election, et cetera…)
   
h is the total number of elections a given pollster has participated in by publishing a poll within at least two months of election day.

(ex. Currently, for Angus-Reid, h = 22, for Harris/Decima, h = 17, et cetera…)
   
Rxk is defined as the election result of a given party, x, as reported by the election authority of the jurisdiction in question for a given election, k. It is expressed as a percentage of total number of valid votes cast. For the purposes of this calculation, federal election results are broken down by province or region such that any given federal election is considered to comprise separate six separate ‘Rx values, one each for Ontario, Quebec, British Columbia, Alberta, the Prairie region (Manitoba and Saskatchewan) and Atlantic Canada (Nova Scotia, PEI, New Brunswick and Newfoundland and Labrador).
   
axk is defined as the predicted election result for a given party, x, as determined by the final poll published by the polling firm ‘a‘ before a given election, k.

(ex. The predicted election result of the Manitoba New Democratic Party published in the last poll by the Angus Reid polling firm before a specific Manitoba general election.)

Relative Error Score methodology:

For a given pollster, ‘a‘, whereas ‘x’ represents a given political party:

For pollster ‘a‘:

Whereas:

Ca is defined as the Consistency Score for pollster ‘a‘;
Cb is defined as the Consistency Score for pollster ‘b‘; and
Cz is defined as the Consistency Score for the zth pollster
   
Sa is defined as the Average Error Score for pollster ‘a‘;
Sb is defined as the Average Error Score for pollster ‘b‘; and
Sz is defined as the Average Error Score for the zth pollster
   
Wa is defined as the Weighted Error Score for pollster ‘a‘;
Wb is defined as the Weighted Error Score for pollster ‘b‘; and
Wz is defined as the Weighted Error Score for the zth pollster
   
Ea is defined as the Relative Error Score for pollster ‘a‘;
Eb is defined as the Relative Error Score for pollster ‘b‘; and
Ez is defined as the Relative Error Score for the zth pollster
   
x is defined as a given political party. See “n” for more information.

(ex. x = 5 = Green Party or x = 1 = Conservative Party)
   
n is the total number of political parties a given pollster has evaluated by publishing a poll within two months of an election. For the purposes of these calculations, all provincial New Democratic Parties and the federal NDP are considered to be of the same political party. The Bloc Québécois and the Parti Québécois are also considered to be of the same political party. All provincial liberal parties are also considered to be of the same political party as the Liberal Party of Canada (including officially non-aligned and independent provincial parties such as the Liberal Party of Ontario and the Parti Libéral du Québec). All provincial green parties are considered to be of the same political party as the Green Party of Canada. All provincial Progressive Conservative parties, the British Columbia Conservative Party and the Saskatchewan Party are considered to be of the same political party as the Conservative Party of Canada. The Wildrose Party of Alberta is considered as its own entity and is not considered aligned with any other political party. Independents are not included in this calculation.
   
k is defined as any given election.

(ex. 2011 Ontario General Election or the 2009 Nova Scotia General Election, et cetera…)
   
h is the total number of elections a given pollster has participated in by publishing a poll within at least two months of election day.

(ex. Currently, for Angus-Reid, h = 22, for Harris/Decima, h = 17, et cetera…)
   
Rxk is defined as the election result of a given party, x, as reported by the election authority of the jurisdiction in question for a given election, k. It is expressed as a percentage of total number of valid votes cast. For the purposes of this calculation, federal election results are broken down by province or region such that any given federal election is considered to comprise separate six separate ‘Rx values, one each for Ontario, Quebec, British Columbia, Alberta, the Prairie region (Manitoba and Saskatchewan) and Atlantic Canada (Nova Scotia, PEI, New Brunswick and Newfoundland and Labrador).
   
axk is defined as the predicted election result for a given party, x, as determined by the final poll published by the polling firm ‘a‘ before a given election, k.

(ex. The predicted election result of the Manitoba New Democratic Party published in the last poll by the Angus Reid polling firm before a specific Manitoba general election.)

Weighted Pollster Score methodology:

The weighted pollster ranking methodology shall remain largely proprietary.

On a very general level, it employs the same initial step as the Relative Error Score methodology, namely:

For a given pollster, ‘a‘, whereas ‘x’ represents a given political party:

It then weights each Wa1 value based on three factors:

  1. The length of time since the election was held.
  2. The size of the electorate (ex. An Ontario election wold receive a larger weight than a BC election held on the same date; an Alberta election would receive a larger weight than a PEI election held on the same date; etc.)
  3. How much difficulty a given polling firms had accurately predicting the outcome of a given election compared to the difficulty experienced by competing polling firms. This measure is based on the size of the standard deviation of the total error for each polling firm for that election.

 

Overall Pollster Score methodology:

For a given pollster, ‘a‘:

Whereas:

Ca is defined as the Consistency Score for pollster ‘a‘;
Sa is defined as the Average Error Score for pollster ‘a‘;
Wa is defined as the Weighted Error Score for pollster ‘a‘;
Ea is defined as the Relative Error Score for pollster ‘a‘;