Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 237 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.5564. Min Act: -0.0559

Data Index: 6477888 (Open Web Text)

Max Activating Token Index: 646

Text #1

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4627. Min Act: -0.0862

Data Index: 7250487 (Open Web Text)

Max Activating Token Index: 679

Text #2

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4383. Min Act: -0.0474

Data Index: 8139689 (Open Web Text)

Max Activating Token Index: 826

Text #3

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4367. Min Act: -0.0570

Data Index: 8664234 (Open Web Text)

Max Activating Token Index: 594

Text #4

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4238. Min Act: -0.1460

Data Index: 2365231 (Open Web Text)

Max Activating Token Index: 659

Text #5

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4306. Min Act: -0.0385

Data Index: 381397 (Open Web Text)

Max Activating Token Index: 492

Text #6

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4208. Min Act: -0.0905

Data Index: 1757329 (Open Web Text)

Max Activating Token Index: 6

Text #7

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4200. Min Act: -0.1148

Data Index: 2655787 (Open Web Text)

Max Activating Token Index: 122

Text #8

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4094. Min Act: -0.0250

Data Index: 8720748 (Open Web Text)

Max Activating Token Index: 808

Text #9

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4085. Min Act: -0.0153

Data Index: 1267444 (Open Web Text)

Max Activating Token Index: 820

Text #10

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4095. Min Act: -0.0667

Data Index: 2107457 (Open Web Text)

Max Activating Token Index: 43

Text #11

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4044. Min Act: -0.0832

Data Index: 6561459 (Open Web Text)

Max Activating Token Index: 697

Text #12

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4139. Min Act: -0.0569

Data Index: 351454 (Open Web Text)

Max Activating Token Index: 348

Text #13

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4076. Min Act: -0.0374

Data Index: 926246 (Open Web Text)

Max Activating Token Index: 671

Text #14

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4102. Min Act: -0.0844

Data Index: 2642158 (Open Web Text)

Max Activating Token Index: 836

Text #15

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4155. Min Act: -0.0904

Data Index: 2197448 (Open Web Text)

Max Activating Token Index: 62

Text #16

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.3976. Min Act: -0.0750

Data Index: 8633210 (Open Web Text)

Max Activating Token Index: 135

Text #17

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.4034. Min Act: -0.0951

Data Index: 6848801 (Open Web Text)

Max Activating Token Index: 741

Text #18

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.3953. Min Act: -0.0998

Data Index: 1612287 (Open Web Text)

Max Activating Token Index: 153

Text #19

Max Range: 1.5564. Min Range: -1.5564

Max Act: 1.3984. Min Act: -0.1312

Data Index: 8686377 (Open Web Text)

Max Activating Token Index: 855