Blocking experimental data

You are working with a manufacturing firm that wants to conduct some experiments on worker productivity. Their dataset only contains 100 rows, so it's important that experimental groups are balanced.

This sounds like a great opportunity to use your knowledge of blocking to assist them. They have provided a productivity_subjects DataFrame. Split the provided dataset into two even groups of 50 entries each.

The libraries numpy and pandas have been imported as np and pd respectively.

This exercise is part of the course

Experimental Design in Python

Exercise instructions

Randomly select 50 subjects from the productivity_subjects DataFrame into a new DataFrame block_1 without replacement.
Set a new column, block to 1 for the block_1 DataFrame.
Assign the remaining subjects to a DataFrame called block_2 and set the block column to 2 for this DataFrame.
Concatenate the blocks together into a single DataFrame, and print the count of each value in the block column to confirm the blocking worked.

Hands-on interactive exercise

Have a go at this exercise by completing this sample code.

# Randomly assign half
block_1 = productivity_subjects.____(____, random_state=42, ____)

# Set the block column
block_1['block'] = ____

# Create second assignment and label
block_2 = ____
block_2['block'] = ____

# Concatenate and print
productivity_combined = pd.____([block_1, block_2], axis=0)
print(productivity_combined['block'].value_counts())

Edit and Run Code

This exercise is part of the course

Experimental Design in Python

IntermediateSkill Level

4.8+

Start Course for Free

Building knowledge in experimental design allows you to test hypotheses with best-practice analytical tools and quantify the risk of your work. You’ll begin your journey by setting the foundations of what experimental design is and different experimental design setups such as blocking and stratification. You’ll then learn and apply visual and analytical tests for normality in experimental data.

Exercise 1: Setting up experiments Exercise 2: Non-random assignment of subjects Exercise 3: Random assignment of subjects Exercise 4: Experimental data setup Exercise 5: Blocking experimental data

Current Exercise

Exercise 6: Stratifying an experiment Exercise 7: Which was stratified?Exercise 8: Normal data Exercise 9: Visual normality in an agricultural experiment Exercise 10: Analytical normality in an agricultural experiment

You'll delve into sophisticated experimental design techniques, focusing on factorial designs, randomized block designs, and covariate adjustments. These methodologies are instrumental in enhancing the accuracy, efficiency, and interpretability of experimental results. Through a combination of theoretical insights and practical applications, you'll acquire the skills needed to design, implement, and analyze complex experiments in various fields of research.

Exercise 1: Factorial designs: principles and applications Exercise 2: Understanding marketing campaign effectiveness Exercise 3: Heatmap of campaign interactions Exercise 4: Factorial designs and randomized block designs Exercise 5: Randomized block design: controlling variance Exercise 6: Implementing a randomized block design Exercise 7: Visualizing productivity within blocks by incentive Exercise 8: ANOVA within blocks of employees Exercise 9: Covariate adjustment in experimental design Exercise 10: Importance of covariates Exercise 11: Covariate adjustment with chick growth

Master statistical tests like t-tests, ANOVA, and Chi-Square, and dive deep into post-hoc analyses and power analysis essentials. Learn to select the right test, interpret p-values and errors, and skillfully conduct power analysis to determine sample and effect sizes, all while leveraging Python's powerful libraries to bring your data insights to life.

Exercise 1: Choosing the right statistical test Exercise 2: Choosing the right test: petrochemicals Exercise 3: Choosing the right test: human resources Exercise 4: Choosing the right test: finance Exercise 5: Post-hoc analysis following ANOVA Exercise 6: Anxiety treatments ANOVA Exercise 7: Applying Tukey's HSD Exercise 8: Applying Bonferoni correction Exercise 9: P-values, alpha, and errors Exercise 10: Analyzing toy durability Exercise 11: Visualizing durability differences Exercise 12: Role of significance levels Exercise 13: Power analysis: sample and effect size Exercise 14: Effect size purpose Exercise 15: Estimating required sample size for energy study

Hop into the complexities of experimental data analysis. Learn to synthesize insights using pandas, address data issues like heteroscedasticity with scipy.stats, and apply nonparametric tests like Mann-Whitney U. Learn additional techniques for transforming, visualizing, and interpreting complex data, enhancing your ability to conduct robust analyses in various experimental settings.

Exercise 1: Synthesizing insights from complex experiments Exercise 2: Visualizing loan approval yield Exercise 3: Exploring customer satisfaction Exercise 4: Effectively communicating experimental data Exercise 5: Addressing complexities in experimental data Exercise 6: Check for heteroscedasticity in shelf life Exercise 7: Exploring and transforming shelf life data Exercise 8: Applying nonparametric tests in experimental analysis Exercise 9: Visualizing and testing preservation methods Exercise 10: Further analyzing food preservation techniques Exercise 11: Congratulations!