Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2891 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0411. Min Range: -3.0411

Max Act: 3.0411. Min Act: -0.1700

Data Index: 4719453 (Open Web Text)

Max Activating Token Index: 708

Text #1

Max Range: 3.0411. Min Range: -3.0411

Max Act: 3.0047. Min Act: -0.1700

Data Index: 3604240 (Open Web Text)

Max Activating Token Index: 685

Text #2

Max Range: 3.0411. Min Range: -3.0411

Max Act: 3.0134. Min Act: -0.1700

Data Index: 1153863 (Open Web Text)

Max Activating Token Index: 664

Text #3

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9754. Min Act: -0.1700

Data Index: 3262718 (Open Web Text)

Max Activating Token Index: 102

Text #4

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9832. Min Act: -0.1700

Data Index: 7845276 (Open Web Text)

Max Activating Token Index: 869

Text #5

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9760. Min Act: -0.1700

Data Index: 8153581 (Open Web Text)

Max Activating Token Index: 970

Text #6

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9713. Min Act: -0.1700

Data Index: 6910808 (Open Web Text)

Max Activating Token Index: 251

Text #7

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9388. Min Act: -0.1700

Data Index: 6009536 (Open Web Text)

Max Activating Token Index: 274

Text #8

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9372. Min Act: -0.1700

Data Index: 8039716 (Open Web Text)

Max Activating Token Index: 1006

Text #9

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9257. Min Act: -0.1700

Data Index: 2604358 (Open Web Text)

Max Activating Token Index: 213

Text #10

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9434. Min Act: -0.1700

Data Index: 1534419 (Open Web Text)

Max Activating Token Index: 158

Text #11

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9349. Min Act: -0.1700

Data Index: 7241920 (Open Web Text)

Max Activating Token Index: 631

Text #12

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9112. Min Act: -0.1700

Data Index: 3804088 (Open Web Text)

Max Activating Token Index: 416

Text #13

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9337. Min Act: -0.1700

Data Index: 671289 (Open Web Text)

Max Activating Token Index: 159

Text #14

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9084. Min Act: -0.1700

Data Index: 7339167 (Open Web Text)

Max Activating Token Index: 292

Text #15

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9191. Min Act: -0.1700

Data Index: 3915545 (Open Web Text)

Max Activating Token Index: 944

Text #16

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9328. Min Act: -0.1700

Data Index: 7137282 (Open Web Text)

Max Activating Token Index: 383

Text #17

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.9050. Min Act: -0.1700

Data Index: 3746600 (Open Web Text)

Max Activating Token Index: 87

Text #18

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.8862. Min Act: -0.1700

Data Index: 3136309 (Open Web Text)

Max Activating Token Index: 316

Text #19

Max Range: 3.0411. Min Range: -3.0411

Max Act: 2.8673. Min Act: -0.1700

Data Index: 2372081 (Open Web Text)

Max Activating Token Index: 707