Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1306 in Layer 0

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.0067. Min Range: -1.0067

Max Act: 1.0061. Min Act: -0.0080

Data Index: 3647711 (Open Web Text)

Max Activating Token Index: 500

Text #1

Max Range: 1.0067. Min Range: -1.0067

Max Act: 1.0067. Min Act: -0.0615

Data Index: 3647710 (Open Web Text)

Max Activating Token Index: 512

Text #2

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9935. Min Act: -0.1447

Data Index: 6624213 (Open Web Text)

Max Activating Token Index: 491

Text #3

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9892. Min Act: -0.0966

Data Index: 3647712 (Open Web Text)

Max Activating Token Index: 483

Text #4

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9828. Min Act: -0.1001

Data Index: 5793381 (Open Web Text)

Max Activating Token Index: 500

Text #5

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9855. Min Act: -0.1083

Data Index: 1692073 (Open Web Text)

Max Activating Token Index: 511

Text #6

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9830. Min Act: 0.1033

Data Index: 3647709 (Open Web Text)

Max Activating Token Index: 523

Text #7

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9692. Min Act: -0.0882

Data Index: 83907 (Open Web Text)

Max Activating Token Index: 473

Text #8

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9714. Min Act: -0.0868

Data Index: 5612952 (Open Web Text)

Max Activating Token Index: 511

Text #9

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9716. Min Act: -0.0905

Data Index: 4533103 (Open Web Text)

Max Activating Token Index: 507

Text #10

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9673. Min Act: -0.1020

Data Index: 7854852 (Open Web Text)

Max Activating Token Index: 506

Text #11

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9566. Min Act: -0.0630

Data Index: 444599 (Open Web Text)

Max Activating Token Index: 484

Text #12

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9617. Min Act: -0.1182

Data Index: 4576902 (Open Web Text)

Max Activating Token Index: 497

Text #13

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9576. Min Act: -0.0769

Data Index: 4453175 (Open Web Text)

Max Activating Token Index: 428

Text #14

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9520. Min Act: -0.1309

Data Index: 4083300 (Open Web Text)

Max Activating Token Index: 517

Text #15

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9568. Min Act: -0.1441

Data Index: 7093319 (Open Web Text)

Max Activating Token Index: 455

Text #16

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9582. Min Act: -0.0791

Data Index: 198571 (Open Web Text)

Max Activating Token Index: 497

Text #17

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9571. Min Act: -0.0696

Data Index: 5612951 (Open Web Text)

Max Activating Token Index: 974

Text #18

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9507. Min Act: -0.1307

Data Index: 2525897 (Open Web Text)

Max Activating Token Index: 520

Text #19

Max Range: 1.0067. Min Range: -1.0067

Max Act: 0.9492. Min Act: -0.1317

Data Index: 2212002 (Open Web Text)

Max Activating Token Index: 515