Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 230 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 8.4142. Min Range: -8.4142

Max Act: 8.4142. Min Act: -0.1700

Data Index: 791187 (Open Web Text)

Max Activating Token Index: 57

Text #1

Max Range: 8.4142. Min Range: -8.4142

Max Act: 8.3427. Min Act: -0.1700

Data Index: 6504686 (Open Web Text)

Max Activating Token Index: 553

Text #2

Max Range: 8.4142. Min Range: -8.4142

Max Act: 8.2434. Min Act: -0.1700

Data Index: 6426180 (Open Web Text)

Max Activating Token Index: 990

Text #3

Max Range: 8.4142. Min Range: -8.4142

Max Act: 8.1289. Min Act: -0.1700

Data Index: 4194828 (Open Web Text)

Max Activating Token Index: 453

Text #4

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.9397. Min Act: -0.1700

Data Index: 4766197 (Open Web Text)

Max Activating Token Index: 730

Text #5

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.8892. Min Act: -0.1700

Data Index: 5425042 (Open Web Text)

Max Activating Token Index: 829

Text #6

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.9884. Min Act: -0.1700

Data Index: 8489811 (Open Web Text)

Max Activating Token Index: 487

Text #7

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.9306. Min Act: -0.1700

Data Index: 1201433 (Open Web Text)

Max Activating Token Index: 408

Text #8

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.9448. Min Act: -0.1700

Data Index: 3016327 (Open Web Text)

Max Activating Token Index: 724

Text #9

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.8280. Min Act: -0.1700

Data Index: 432280 (Open Web Text)

Max Activating Token Index: 527

Text #10

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.8109. Min Act: -0.1700

Data Index: 5466702 (Open Web Text)

Max Activating Token Index: 994

Text #11

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.8281. Min Act: -0.1700

Data Index: 1582225 (Open Web Text)

Max Activating Token Index: 880

Text #12

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.7991. Min Act: -0.1700

Data Index: 213302 (Open Web Text)

Max Activating Token Index: 294

Text #13

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.8663. Min Act: -0.1700

Data Index: 4065491 (Open Web Text)

Max Activating Token Index: 553

Text #14

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.7951. Min Act: -0.1700

Data Index: 5467417 (Open Web Text)

Max Activating Token Index: 506

Text #15

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.7461. Min Act: -0.1700

Data Index: 3509554 (Open Web Text)

Max Activating Token Index: 456

Text #16

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.7376. Min Act: -0.1700

Data Index: 1136780 (Open Web Text)

Max Activating Token Index: 296

Text #17

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.7639. Min Act: -0.1700

Data Index: 6291112 (Open Web Text)

Max Activating Token Index: 634

Text #18

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.8126. Min Act: -0.1700

Data Index: 4740478 (Open Web Text)

Max Activating Token Index: 267

Text #19

Max Range: 8.4142. Min Range: -8.4142

Max Act: 7.7475. Min Act: -0.1700

Data Index: 1776223 (Open Web Text)

Max Activating Token Index: 978