Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2366 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.2364. Min Act: -0.1700

Data Index: 8423955 (Open Web Text)

Max Activating Token Index: 993

Text #1

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.2213. Min Act: -0.1700

Data Index: 5115300 (Open Web Text)

Max Activating Token Index: 883

Text #2

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.2074. Min Act: -0.1700

Data Index: 8670158 (Open Web Text)

Max Activating Token Index: 857

Text #3

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1581. Min Act: -0.1700

Data Index: 3315831 (Open Web Text)

Max Activating Token Index: 439

Text #4

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1829. Min Act: -0.1700

Data Index: 50365 (Open Web Text)

Max Activating Token Index: 679

Text #5

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1901. Min Act: -0.1700

Data Index: 3353745 (Open Web Text)

Max Activating Token Index: 915

Text #6

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1522. Min Act: -0.1700

Data Index: 1994467 (Open Web Text)

Max Activating Token Index: 994

Text #7

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1429. Min Act: -0.1700

Data Index: 1382501 (Open Web Text)

Max Activating Token Index: 691

Text #8

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1252. Min Act: -0.1700

Data Index: 6956604 (Open Web Text)

Max Activating Token Index: 909

Text #9

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1251. Min Act: -0.1700

Data Index: 515634 (Open Web Text)

Max Activating Token Index: 885

Text #10

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1228. Min Act: -0.1700

Data Index: 3761985 (Open Web Text)

Max Activating Token Index: 672

Text #11

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1171. Min Act: -0.1700

Data Index: 1698302 (Open Web Text)

Max Activating Token Index: 987

Text #12

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.0748. Min Act: -0.1700

Data Index: 8649941 (Open Web Text)

Max Activating Token Index: 337

Text #13

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.1068. Min Act: -0.1700

Data Index: 7297544 (Open Web Text)

Max Activating Token Index: 723

Text #14

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.0983. Min Act: -0.1700

Data Index: 6956605 (Open Web Text)

Max Activating Token Index: 758

Text #15

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.0992. Min Act: -0.1700

Data Index: 2661340 (Open Web Text)

Max Activating Token Index: 477

Text #16

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.0881. Min Act: -0.1700

Data Index: 6557276 (Open Web Text)

Max Activating Token Index: 640

Text #17

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.0619. Min Act: -0.1700

Data Index: 4971638 (Open Web Text)

Max Activating Token Index: 656

Text #18

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.0750. Min Act: -0.1700

Data Index: 5578801 (Open Web Text)

Max Activating Token Index: 572

Text #19

Max Range: 4.2364. Min Range: -4.2364

Max Act: 4.0623. Min Act: -0.1700

Data Index: 5643837 (Open Web Text)

Max Activating Token Index: 638