Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2700 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.1510. Min Act: -0.1700

Data Index: 2440851 (Open Web Text)

Max Activating Token Index: 561

Text #1

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.1494. Min Act: -0.1700

Data Index: 333666 (Open Web Text)

Max Activating Token Index: 415

Text #2

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.0383. Min Act: -0.1700

Data Index: 7699223 (Open Web Text)

Max Activating Token Index: 885

Text #3

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.0375. Min Act: -0.1700

Data Index: 1595630 (Open Web Text)

Max Activating Token Index: 264

Text #4

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9804. Min Act: -0.1700

Data Index: 4688344 (Open Web Text)

Max Activating Token Index: 666

Text #5

Max Range: 3.1510. Min Range: -3.1510

Max Act: 3.0187. Min Act: -0.1700

Data Index: 6183559 (Open Web Text)

Max Activating Token Index: 459

Text #6

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9895. Min Act: -0.1700

Data Index: 3414360 (Open Web Text)

Max Activating Token Index: 232

Text #7

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9356. Min Act: -0.1700

Data Index: 226994 (Open Web Text)

Max Activating Token Index: 777

Text #8

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9547. Min Act: -0.1700

Data Index: 6696219 (Open Web Text)

Max Activating Token Index: 708

Text #9

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9462. Min Act: -0.1700

Data Index: 2865664 (Open Web Text)

Max Activating Token Index: 814

Text #10

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9736. Min Act: -0.1700

Data Index: 7170630 (Open Web Text)

Max Activating Token Index: 812

Text #11

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9404. Min Act: -0.1700

Data Index: 171165 (Open Web Text)

Max Activating Token Index: 982

Text #12

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9310. Min Act: -0.1700

Data Index: 5887749 (Open Web Text)

Max Activating Token Index: 106

Text #13

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9202. Min Act: -0.1700

Data Index: 4108253 (Open Web Text)

Max Activating Token Index: 941

Text #14

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9263. Min Act: -0.1700

Data Index: 46094 (Open Web Text)

Max Activating Token Index: 913

Text #15

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9327. Min Act: -0.1700

Data Index: 4454854 (Open Web Text)

Max Activating Token Index: 379

Text #16

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9211. Min Act: -0.1700

Data Index: 181545 (Open Web Text)

Max Activating Token Index: 140

Text #17

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9335. Min Act: -0.1700

Data Index: 6457651 (Open Web Text)

Max Activating Token Index: 400

Text #18

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9344. Min Act: -0.1700

Data Index: 3077530 (Open Web Text)

Max Activating Token Index: 279

Text #19

Max Range: 3.1510. Min Range: -3.1510

Max Act: 2.9228. Min Act: -0.1700

Data Index: 5244434 (Open Web Text)

Max Activating Token Index: 774