Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 424 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.2369. Min Act: -0.1700

Data Index: 4185542 (Open Web Text)

Max Activating Token Index: 785

Text #1

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1739. Min Act: -0.1700

Data Index: 7009681 (Open Web Text)

Max Activating Token Index: 295

Text #2

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1740. Min Act: -0.1700

Data Index: 5649088 (Open Web Text)

Max Activating Token Index: 830

Text #3

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1035. Min Act: -0.1700

Data Index: 5240038 (Open Web Text)

Max Activating Token Index: 171

Text #4

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1485. Min Act: -0.1700

Data Index: 1598339 (Open Web Text)

Max Activating Token Index: 60

Text #5

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0736. Min Act: -0.1700

Data Index: 6704053 (Open Web Text)

Max Activating Token Index: 758

Text #6

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1297. Min Act: -0.1700

Data Index: 1555059 (Open Web Text)

Max Activating Token Index: 221

Text #7

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1145. Min Act: -0.1700

Data Index: 2677501 (Open Web Text)

Max Activating Token Index: 84

Text #8

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1036. Min Act: -0.1700

Data Index: 1864604 (Open Web Text)

Max Activating Token Index: 374

Text #9

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.1039. Min Act: -0.1700

Data Index: 7009670 (Open Web Text)

Max Activating Token Index: 746

Text #10

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0543. Min Act: -0.1700

Data Index: 8451388 (Open Web Text)

Max Activating Token Index: 513

Text #11

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0790. Min Act: -0.1700

Data Index: 792731 (Open Web Text)

Max Activating Token Index: 5

Text #12

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0982. Min Act: -0.1700

Data Index: 2677508 (Open Web Text)

Max Activating Token Index: 176

Text #13

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0632. Min Act: -0.1700

Data Index: 2677506 (Open Web Text)

Max Activating Token Index: 138

Text #14

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0584. Min Act: -0.1700

Data Index: 2677525 (Open Web Text)

Max Activating Token Index: 494

Text #15

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0452. Min Act: -0.1700

Data Index: 6443985 (Open Web Text)

Max Activating Token Index: 533

Text #16

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0533. Min Act: -0.1700

Data Index: 6215204 (Open Web Text)

Max Activating Token Index: 205

Text #17

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0222. Min Act: -0.1700

Data Index: 302167 (Open Web Text)

Max Activating Token Index: 127

Text #18

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0370. Min Act: -0.1700

Data Index: 5078111 (Open Web Text)

Max Activating Token Index: 59

Text #19

Max Range: 3.2369. Min Range: -3.2369

Max Act: 3.0013. Min Act: -0.1700

Data Index: 1555052 (Open Web Text)

Max Activating Token Index: 347