Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2738 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.1241. Min Act: -0.1700

Data Index: 7400812 (Open Web Text)

Max Activating Token Index: 572

Text #1

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0716. Min Act: -0.1700

Data Index: 1468604 (Open Web Text)

Max Activating Token Index: 465

Text #2

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0569. Min Act: -0.1700

Data Index: 4326960 (Open Web Text)

Max Activating Token Index: 972

Text #3

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0272. Min Act: -0.1700

Data Index: 7300417 (Open Web Text)

Max Activating Token Index: 832

Text #4

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0181. Min Act: -0.1700

Data Index: 5835991 (Open Web Text)

Max Activating Token Index: 285

Text #5

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0445. Min Act: -0.1700

Data Index: 6835723 (Open Web Text)

Max Activating Token Index: 437

Text #6

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0402. Min Act: -0.1700

Data Index: 4923759 (Open Web Text)

Max Activating Token Index: 608

Text #7

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0339. Min Act: -0.1700

Data Index: 177166 (Open Web Text)

Max Activating Token Index: 769

Text #8

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0009. Min Act: -0.1700

Data Index: 1951653 (Open Web Text)

Max Activating Token Index: 411

Text #9

Max Range: 4.1241. Min Range: -4.1241

Max Act: 4.0223. Min Act: -0.1700

Data Index: 6778556 (Open Web Text)

Max Activating Token Index: 154

Text #10

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9628. Min Act: -0.1700

Data Index: 7187737 (Open Web Text)

Max Activating Token Index: 139

Text #11

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9768. Min Act: -0.1700

Data Index: 5154218 (Open Web Text)

Max Activating Token Index: 169

Text #12

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9892. Min Act: -0.1700

Data Index: 4937938 (Open Web Text)

Max Activating Token Index: 77

Text #13

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9698. Min Act: -0.1700

Data Index: 1130660 (Open Web Text)

Max Activating Token Index: 694

Text #14

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9782. Min Act: -0.1700

Data Index: 1089324 (Open Web Text)

Max Activating Token Index: 136

Text #15

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9594. Min Act: -0.1700

Data Index: 3851954 (Open Web Text)

Max Activating Token Index: 236

Text #16

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9645. Min Act: -0.1700

Data Index: 4184285 (Open Web Text)

Max Activating Token Index: 207

Text #17

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9639. Min Act: -0.1700

Data Index: 2226340 (Open Web Text)

Max Activating Token Index: 625

Text #18

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9684. Min Act: -0.1700

Data Index: 5630677 (Open Web Text)

Max Activating Token Index: 824

Text #19

Max Range: 4.1241. Min Range: -4.1241

Max Act: 3.9451. Min Act: -0.1700

Data Index: 5527200 (Open Web Text)

Max Activating Token Index: 192