Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 189 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.3492. Min Act: -0.1700

Data Index: 243357 (Open Web Text)

Max Activating Token Index: 346

Text #1

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.3335. Min Act: -0.1675

Data Index: 8594651 (Open Web Text)

Max Activating Token Index: 634

Text #2

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.3009. Min Act: -0.1700

Data Index: 243355 (Open Web Text)

Max Activating Token Index: 679

Text #3

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.2536. Min Act: -0.1656

Data Index: 5658020 (Open Web Text)

Max Activating Token Index: 174

Text #4

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.2537. Min Act: -0.1700

Data Index: 5517105 (Open Web Text)

Max Activating Token Index: 289

Text #5

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.2527. Min Act: -0.1700

Data Index: 7962501 (Open Web Text)

Max Activating Token Index: 730

Text #6

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.2276. Min Act: -0.1700

Data Index: 243358 (Open Web Text)

Max Activating Token Index: 311

Text #7

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.2344. Min Act: -0.1700

Data Index: 1384232 (Open Web Text)

Max Activating Token Index: 50

Text #8

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.2272. Min Act: -0.1700

Data Index: 3584324 (Open Web Text)

Max Activating Token Index: 62

Text #9

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1997. Min Act: -0.1700

Data Index: 3335277 (Open Web Text)

Max Activating Token Index: 269

Text #10

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1930. Min Act: -0.1700

Data Index: 2361574 (Open Web Text)

Max Activating Token Index: 31

Text #11

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1605. Min Act: -0.1700

Data Index: 1059996 (Open Web Text)

Max Activating Token Index: 521

Text #12

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1496. Min Act: -0.1700

Data Index: 7429671 (Open Web Text)

Max Activating Token Index: 575

Text #13

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1299. Min Act: -0.1700

Data Index: 4670963 (Open Web Text)

Max Activating Token Index: 90

Text #14

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1263. Min Act: -0.1700

Data Index: 1277538 (Open Web Text)

Max Activating Token Index: 23

Text #15

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1322. Min Act: -0.1698

Data Index: 243359 (Open Web Text)

Max Activating Token Index: 282

Text #16

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1298. Min Act: -0.1700

Data Index: 3584328 (Open Web Text)

Max Activating Token Index: 802

Text #17

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1128. Min Act: -0.1700

Data Index: 3975243 (Open Web Text)

Max Activating Token Index: 127

Text #18

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.1059. Min Act: -0.1700

Data Index: 6686175 (Open Web Text)

Max Activating Token Index: 413

Text #19

Max Range: 3.3492. Min Range: -3.3492

Max Act: 3.0853. Min Act: -0.1700

Data Index: 428101 (Open Web Text)

Max Activating Token Index: 695