Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 3013 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1625. Min Range: -3.1625

Max Act: 3.1625. Min Act: -0.1700

Data Index: 2555417 (Open Web Text)

Max Activating Token Index: 81

Text #1

Max Range: 3.1625. Min Range: -3.1625

Max Act: 3.0152. Min Act: -0.1700

Data Index: 8399868 (Open Web Text)

Max Activating Token Index: 650

Text #2

Max Range: 3.1625. Min Range: -3.1625

Max Act: 3.0235. Min Act: -0.1700

Data Index: 3924649 (Open Web Text)

Max Activating Token Index: 862

Text #3

Max Range: 3.1625. Min Range: -3.1625

Max Act: 3.0041. Min Act: -0.1700

Data Index: 4833914 (Open Web Text)

Max Activating Token Index: 280

Text #4

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9586. Min Act: -0.1700

Data Index: 4367332 (Open Web Text)

Max Activating Token Index: 877

Text #5

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9612. Min Act: -0.1700

Data Index: 4284756 (Open Web Text)

Max Activating Token Index: 728

Text #6

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9601. Min Act: -0.1700

Data Index: 3164988 (Open Web Text)

Max Activating Token Index: 85

Text #7

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9422. Min Act: -0.1700

Data Index: 8092462 (Open Web Text)

Max Activating Token Index: 535

Text #8

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9569. Min Act: -0.1700

Data Index: 7065938 (Open Web Text)

Max Activating Token Index: 69

Text #9

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9538. Min Act: -0.1700

Data Index: 1690949 (Open Web Text)

Max Activating Token Index: 407

Text #10

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8679. Min Act: -0.1700

Data Index: 499647 (Open Web Text)

Max Activating Token Index: 1019

Text #11

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9246. Min Act: -0.1700

Data Index: 2782325 (Open Web Text)

Max Activating Token Index: 559

Text #12

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8972. Min Act: -0.1700

Data Index: 2035500 (Open Web Text)

Max Activating Token Index: 68

Text #13

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8883. Min Act: -0.1700

Data Index: 6091828 (Open Web Text)

Max Activating Token Index: 29

Text #14

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.9181. Min Act: -0.1700

Data Index: 1083508 (Open Web Text)

Max Activating Token Index: 412

Text #15

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8715. Min Act: -0.1700

Data Index: 7033239 (Open Web Text)

Max Activating Token Index: 939

Text #16

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8701. Min Act: -0.1700

Data Index: 7324747 (Open Web Text)

Max Activating Token Index: 54

Text #17

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8564. Min Act: -0.1700

Data Index: 4252470 (Open Web Text)

Max Activating Token Index: 807

Text #18

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8278. Min Act: -0.1700

Data Index: 6843583 (Open Web Text)

Max Activating Token Index: 287

Text #19

Max Range: 3.1625. Min Range: -3.1625

Max Act: 2.8409. Min Act: -0.1700

Data Index: 7612634 (Open Web Text)

Max Activating Token Index: 136