EXAM June 2026 - Due date June 5th 23:59

The take-home exam comprises two questions. To pass the exam you have to provide complete answers to Questions 1. For higher grades you have to provide complete answers to both questions. The passing grade is 45, 65 and above is a 4/VG and above 85 is grade 5.

I will give partial credit so it's always worth while attempting each task. However, I don't give credit for running analysis, providing plots and tables but not commenting on your answers or motivating what you did. The key is insights and discussion as to why results look like they do.

You should work alone for this exam. The reports will be checked with Urkund.

"Can I use AI for coding?" It's been my experience that, while AI can be helpful, it is also overconfident in providing answers that are in fact wrong. It takes time to check the output from AI and, depending on your own skillset, this may result in an overall waste of time rather than saving time, especially if you want to connect multiple tasks. Your main task is to analyze and interpret the results. AI might help you with setting up the analysis but less so in interpreting and spinning the analysis in the way I have requested.

So I would say that you should use AI with caution and only for simple coding questions like generating plots, proposed functions calls (which may contain bugs), etc. Don't use AI for generating text.

GOOD LUCK!

• The report should not be longer than 1 page of text + max 1/2 page conclusion/main summary per sub-task (sub-tasks = a,b,c etc).
• You do not have to discuss the methods and assumptions unless this pertains to a result you present - i.e. you don't need to write a "lab report" with introduction, methods, analysis and conclusions. Here, it's only analysis and conclusions that I am interested in.
• Figures and Tables: maximum of 4 figures and 2 tables per sub-task.
• Make sure you provide captions for the tables and figures - these should clearly state what the figure/table contains as well as a 1-2 conclusion statement about what can be learnt from the figure/table. Refer to the figures and tables in the text!
• MOTIVATE YOUR ANSWERS!
• Don't contradict your results! If something is unexpected say so and explain why you thought you would see something different.
• Please think about your presentation. Spell check, grammar check. "The arm-length check": meaning, skim through the report as it appears on the screen - does it look nice and neat or is it messy with mis-alignments, cut-off sentences, plots that are tiny etc?
• No cut and paste code mixed with the text.
• No, jupyter notebooks are not accepted as a report. Preferably use latex (overleaf).
• Yes, you can use bulletpoints to present your findings. This could be especially useful for the subtask conclusion.
• Stress your main findings in the 1/2 page subtask conclusion.
• !!!!Make sure you put the figures with each subtask after the subtask discussion and before the next subtask so I don't have to scroll back and forth between tasks when I grade.!!!

IMPORTANT

Wherever the exam task contains a question ("Does X differ from Y?") it is not sufficient to answer "Yes" or "No" - you need to explain how you arrived to this conclusion, which results provide you with these answers, as well as a statement trying explain why the results are what they are. Likewise, when a task is phrased as "Can you do X to get Y?" I, of course, mean for you to attempt to answer this question by doing something, not just hypothesizing about it.

To answer the questions it is for the most part inadequate to perform the task once on the full data set. Make use of re-sampling techniques to support your findings.

Data sets

You will be working on your own data sets that I have put in a quiz on canvas. Everyone's data set is a bit different, by construction. The zip file contains three files: X_TR, y_TR and X_TE, that is, a training data with labels and a separate test set without labels.

Now, let's get started!

Question 1 (70p)

1 (a) (20p)

Construct a pipeline to analyze the X_TR, y_TR data set. Make sure you motivate choices made in your pipeline. Investigate the effect of sample size on your results.

Make sure to discuss:

Overall and class-specific performance of methods and how you assess this

Why you think some methods perform better than others on this data set and for different sample sizes

Particular challenges in this data set that your analysis identified and how you adressed them

How many features were informative and if this dependended on the method and how you assessed this

1 (b) (20p)

What's a minimum disruption to your data set that will make some methods perform worse? This should not be just adding noise to the data because that would be the same for all data sets - what is special about your data? Be more precise in how you target your data and target your chosen methods.

Implement a disruption technique and illustrate how some methods fail and others are more resilient. Motivate why you chose this specific disruption for you data.

Make sure to discuss

How you chose your disruption approach and why

Effect on method performance and explain why this is the expected effect

Rerun multiple times to make sure your assessment is stable

1 (c) (10p)

Is there any evidence of mislabeling in your data set?

Make sure to discuss

How you identified (if any) potentially mislabeled observations

Verify that this technique works by adding mislabeled observations to your data and showing that you can detect them using this technique

1 (d) (20p)

Apply your top-2 best performing methods from 1 (a) to the test data X_TE. Can you say something about how confident you are in your predictions. Be aware that there may be a data drift in X_TE, a change in content distribution compared with X_TR.

Make sure to discuss

How you provide confidence to your predictions on X_TE. Back this up by demonstrating how you did this for test-splits in X_TR

Summarize the prediction results overall and at the class level and discuss what this tells you

How this is affected by the sample size used for training

If you see any evidence of drift, or lack there-of, in X_TE and how you assessed this

Question 2 (30p)

2 (a) (15p)

Use your data and create a much more challenging problem where you have one supermajority class (say 1000+ observations) and the rest are tiny minority classes (you choose - but push the limit so this truly is extreme). How does your pipeline perform on this altered data set? Can you alleviate the problem by weighting or oversampling?

Make sure to discuss

Classification metrics

Effect on performance and why you think this is

2 (b) (15p)

One way to approach this kind of problem is to first run majority-vs-rest and then a multiclass problem for the minority classes. Implement this and investigate if that works for your data set.

Make sure to discuss

Your new pipeline - how well you perform this hiearchical classification

Which methods you chose for the two levels

How well this worked for your data set