Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1309 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1290. Min Range: -5.1290

Max Act: 5.1290. Min Act: -0.1700

Data Index: 7123760 (Open Web Text)

Max Activating Token Index: 859

Text #1

Max Range: 5.1290. Min Range: -5.1290

Max Act: 5.0582. Min Act: -0.1700

Data Index: 5161073 (Open Web Text)

Max Activating Token Index: 141

Text #2

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.9386. Min Act: -0.1700

Data Index: 7123745 (Open Web Text)

Max Activating Token Index: 356

Text #3

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.8752. Min Act: -0.1700

Data Index: 2220549 (Open Web Text)

Max Activating Token Index: 108

Text #4

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.7136. Min Act: -0.1698

Data Index: 846501 (Open Web Text)

Max Activating Token Index: 321

Text #5

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.6365. Min Act: -0.1700

Data Index: 4648097 (Open Web Text)

Max Activating Token Index: 791

Text #6

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.6714. Min Act: -0.1661

Data Index: 483805 (Open Web Text)

Max Activating Token Index: 1012

Text #7

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.6267. Min Act: -0.1674

Data Index: 7218983 (Open Web Text)

Max Activating Token Index: 922

Text #8

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.6275. Min Act: -0.1700

Data Index: 7123749 (Open Web Text)

Max Activating Token Index: 1000

Text #9

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.5429. Min Act: -0.1700

Data Index: 2794150 (Open Web Text)

Max Activating Token Index: 849

Text #10

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.4908. Min Act: -0.1700

Data Index: 654815 (Open Web Text)

Max Activating Token Index: 567

Text #11

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.4710. Min Act: -0.1700

Data Index: 7123761 (Open Web Text)

Max Activating Token Index: 43

Text #12

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.4433. Min Act: -0.1700

Data Index: 5901939 (Open Web Text)

Max Activating Token Index: 520

Text #13

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.3930. Min Act: -0.1700

Data Index: 6612116 (Open Web Text)

Max Activating Token Index: 421

Text #14

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.4347. Min Act: -0.1700

Data Index: 2416234 (Open Web Text)

Max Activating Token Index: 1009

Text #15

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.3730. Min Act: -0.1700

Data Index: 6612111 (Open Web Text)

Max Activating Token Index: 400

Text #16

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.3438. Min Act: -0.1699

Data Index: 5080597 (Open Web Text)

Max Activating Token Index: 794

Text #17

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.3380. Min Act: -0.1700

Data Index: 2740258 (Open Web Text)

Max Activating Token Index: 534

Text #18

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.2965. Min Act: -0.1700

Data Index: 4155667 (Open Web Text)

Max Activating Token Index: 378

Text #19

Max Range: 5.1290. Min Range: -5.1290

Max Act: 4.2916. Min Act: -0.1700

Data Index: 5086612 (Open Web Text)

Max Activating Token Index: 424