Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1374 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.6876. Min Act: -0.1700

Data Index: 2394626 (Open Web Text)

Max Activating Token Index: 1008

Text #1

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.7014. Min Act: -0.1700

Data Index: 5605277 (Open Web Text)

Max Activating Token Index: 440

Text #2

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.6505. Min Act: -0.1700

Data Index: 4079493 (Open Web Text)

Max Activating Token Index: 176

Text #3

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.6774. Min Act: -0.1700

Data Index: 8535702 (Open Web Text)

Max Activating Token Index: 1019

Text #4

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.6319. Min Act: -0.1700

Data Index: 4020588 (Open Web Text)

Max Activating Token Index: 380

Text #5

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.6410. Min Act: -0.1700

Data Index: 4972308 (Open Web Text)

Max Activating Token Index: 149

Text #6

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5926. Min Act: -0.1700

Data Index: 3564675 (Open Web Text)

Max Activating Token Index: 725

Text #7

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5680. Min Act: -0.1700

Data Index: 4200365 (Open Web Text)

Max Activating Token Index: 1005

Text #8

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5718. Min Act: -0.1700

Data Index: 3375493 (Open Web Text)

Max Activating Token Index: 621

Text #9

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5622. Min Act: -0.1700

Data Index: 1407301 (Open Web Text)

Max Activating Token Index: 575

Text #10

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5307. Min Act: -0.1700

Data Index: 5047835 (Open Web Text)

Max Activating Token Index: 484

Text #11

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5311. Min Act: -0.1700

Data Index: 558184 (Open Web Text)

Max Activating Token Index: 379

Text #12

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5191. Min Act: -0.1700

Data Index: 7944534 (Open Web Text)

Max Activating Token Index: 705

Text #13

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5274. Min Act: -0.1700

Data Index: 168852 (Open Web Text)

Max Activating Token Index: 219

Text #14

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.4901. Min Act: -0.1700

Data Index: 1171660 (Open Web Text)

Max Activating Token Index: 649

Text #15

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5123. Min Act: -0.1700

Data Index: 3446515 (Open Web Text)

Max Activating Token Index: 775

Text #16

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.4949. Min Act: -0.1700

Data Index: 5605273 (Open Web Text)

Max Activating Token Index: 662

Text #17

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.4942. Min Act: -0.1700

Data Index: 1720256 (Open Web Text)

Max Activating Token Index: 97

Text #18

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.5085. Min Act: -0.1700

Data Index: 3766453 (Open Web Text)

Max Activating Token Index: 631

Text #19

Max Range: 4.7014. Min Range: -4.7014

Max Act: 4.4809. Min Act: -0.1700

Data Index: 1954728 (Open Web Text)

Max Activating Token Index: 414