Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 94 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.2075. Min Act: -0.1700

Data Index: 7599603 (Open Web Text)

Max Activating Token Index: 137

Text #1

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1951. Min Act: -0.1700

Data Index: 2622315 (Open Web Text)

Max Activating Token Index: 511

Text #2

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1888. Min Act: -0.1700

Data Index: 2622316 (Open Web Text)

Max Activating Token Index: 247

Text #3

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1670. Min Act: -0.1700

Data Index: 8233301 (Open Web Text)

Max Activating Token Index: 135

Text #4

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1415. Min Act: -0.1700

Data Index: 6663958 (Open Web Text)

Max Activating Token Index: 4

Text #5

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1297. Min Act: -0.1700

Data Index: 4492583 (Open Web Text)

Max Activating Token Index: 201

Text #6

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1346. Min Act: -0.1700

Data Index: 4408874 (Open Web Text)

Max Activating Token Index: 145

Text #7

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1264. Min Act: -0.1700

Data Index: 1227340 (Open Web Text)

Max Activating Token Index: 18

Text #8

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1346. Min Act: -0.1700

Data Index: 4002028 (Open Web Text)

Max Activating Token Index: 721

Text #9

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1186. Min Act: -0.1700

Data Index: 478963 (Open Web Text)

Max Activating Token Index: 7

Text #10

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1095. Min Act: -0.1700

Data Index: 3528043 (Open Web Text)

Max Activating Token Index: 9

Text #11

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1132. Min Act: -0.1700

Data Index: 8367582 (Open Web Text)

Max Activating Token Index: 458

Text #12

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1139. Min Act: -0.1700

Data Index: 3670821 (Open Web Text)

Max Activating Token Index: 572

Text #13

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1121. Min Act: -0.1700

Data Index: 2920246 (Open Web Text)

Max Activating Token Index: 206

Text #14

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1070. Min Act: -0.1700

Data Index: 6612910 (Open Web Text)

Max Activating Token Index: 103

Text #15

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1180. Min Act: -0.1700

Data Index: 3214661 (Open Web Text)

Max Activating Token Index: 401

Text #16

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1070. Min Act: -0.1700

Data Index: 6574803 (Open Web Text)

Max Activating Token Index: 16

Text #17

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.1019. Min Act: -0.1700

Data Index: 2839170 (Open Web Text)

Max Activating Token Index: 4

Text #18

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.0958. Min Act: -0.1700

Data Index: 1857268 (Open Web Text)

Max Activating Token Index: 259

Text #19

Max Range: 3.2075. Min Range: -3.2075

Max Act: 3.0914. Min Act: -0.1700

Data Index: 2973068 (Open Web Text)

Max Activating Token Index: 8