1351395 - Write a test plan for the "new-profile" ping

Status: RESOLVED FIXED

(Toolkit :: Telemetry, enhancement, P3)

Description

[Alessio Placitelli [:Dexter]]

Before landing the "install" ping, we should make sure to have a solid QA plan. This bug is about writing the test plan that will be executed on the implementation bug.

Comment 1

[Alessio Placitelli [:Dexter]]

Madalin, are you the right person to ask for support on this task? The "new-profile" ping is probably landing (disabled) next week and we need to properly validate it before enabling it.

Comment 2

[Madalin Cotetiu]

Yes I will work on this, I already started creating some test cases for this task and I will share those with you when are ready. Regarding the test plan, do you guys think we need a wiki page for this or we can use a google doc/etherpad?

Comment 3

[Alessio Placitelli [:Dexter]]

(In reply to Madalin Cotetiu from comment #2)
> Yes I will work on this, I already started creating some test cases for this
> task and I will share those with you when are ready. Regarding the test
> plan, do you guys think we need a wiki page for this or we can use a google
> doc/etherpad?

A google doc is fine, thanks!

Comment 4

[Alessio Placitelli [:Dexter]]

The test-plan lives here: https://docs.google.com/document/d/1THuy92VmgzNx7ioYgAP3e8jkUQn8CWShRm1HqcNE4Ho/edit

Georg, does this look ok to you? Can you think of anything else that was not covered?

Comment 5

[Alessio Placitelli [:Dexter]]

Moving the ni? to Chris (thanks!).

Comment 6

[Chris H-C :chutten|PTO (back Oct 23)]

Maybe a case or two around ensuring that the ping is sent at most once. A "Ping sent, Firefox restart, ensure ping isn't sent." kinda thing?

Looks pretty comprehensive to me.

Comment 7

[Madalin Cotetiu]

We have that case: 6. Check if restarting firefox after the new profile ping is sent does not sent a new ping. and 17. Check if the ping is not sent after a profile refresh (new profile ping already sent before the refresh occurred)

I will also add a test case involving a crash and a test when the user waits 30 minutes after the new profile ping is sent cheeking that the ping is no longer sent.

Comment 8

[Alessio Placitelli [:Dexter]]

Cool, thanks Chris! It looks like we can close this bug, as the test-plan was written and it's being executed on bug 1120370.

Comment 9

[Georg Fritzsche [:gfritzsche]]

(In reply to Alessio Placitelli [:Dexter] from comment #4)
> The test-plan lives here:
> https://docs.google.com/document/d/
> 1THuy92VmgzNx7ioYgAP3e8jkUQn8CWShRm1HqcNE4Ho/edit
> 
> Georg, does this look ok to you? Can you think of anything else that was not
> covered?

I left some comments on the doc that we should clear up.