Building the question pool
A knowledge quiz with the same ten questions for all participants is worthless after the first round — the answers circulate in internal Slack within two days. A question pool of thirty to fifty questions from which ten are randomly drawn keeps the quiz robust against sharing and allows repetitions without identity detection.
For the build: every question gets a difficulty level (easy, medium, hard) and a topic category. At quiz start the calculation engine randomly draws a mix of categories and difficulties — e.g. "three easy, five medium, two hard, evenly distributed across four topics". This way all participants get a comparable quiz but not an identical one.
Question quality matters. Unambiguously phrased questions with a clearly correct answer are the goal — questions where "it depends" would be a plausible answer belong in a discussion thread, not the quiz. Before going live all questions should be reviewed by two to three domain experts. An ambiguous question distorts the score distribution and undermines trust in the certificate.