Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2914 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.6954. Min Act: -0.1503

Data Index: 2296698 (Open Web Text)

Max Activating Token Index: 85

Text #1

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.6479. Min Act: -0.1700

Data Index: 5801523 (Open Web Text)

Max Activating Token Index: 918

Text #2

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.6246. Min Act: -0.1700

Data Index: 5801530 (Open Web Text)

Max Activating Token Index: 1009

Text #3

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.6243. Min Act: -0.1495

Data Index: 2296709 (Open Web Text)

Max Activating Token Index: 226

Text #4

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.6201. Min Act: -0.1241

Data Index: 7533335 (Open Web Text)

Max Activating Token Index: 264

Text #5

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.6039. Min Act: -0.1700

Data Index: 2150987 (Open Web Text)

Max Activating Token Index: 71

Text #6

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5978. Min Act: -0.1363

Data Index: 7533349 (Open Web Text)

Max Activating Token Index: 403

Text #7

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5974. Min Act: -0.1700

Data Index: 6249085 (Open Web Text)

Max Activating Token Index: 269

Text #8

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5897. Min Act: -0.1687

Data Index: 3094841 (Open Web Text)

Max Activating Token Index: 743

Text #9

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5836. Min Act: -0.1552

Data Index: 2134566 (Open Web Text)

Max Activating Token Index: 39

Text #10

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5837. Min Act: -0.1700

Data Index: 2758216 (Open Web Text)

Max Activating Token Index: 988

Text #11

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5848. Min Act: -0.1305

Data Index: 2296702 (Open Web Text)

Max Activating Token Index: 218

Text #12

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5865. Min Act: -0.1700

Data Index: 4139863 (Open Web Text)

Max Activating Token Index: 675

Text #13

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5723. Min Act: -0.1700

Data Index: 8070640 (Open Web Text)

Max Activating Token Index: 202

Text #14

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5608. Min Act: -0.1700

Data Index: 7661514 (Open Web Text)

Max Activating Token Index: 665

Text #15

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5840. Min Act: -0.1700

Data Index: 1220730 (Open Web Text)

Max Activating Token Index: 43

Text #16

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5724. Min Act: -0.1700

Data Index: 3460316 (Open Web Text)

Max Activating Token Index: 267

Text #17

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5703. Min Act: -0.1700

Data Index: 7500472 (Open Web Text)

Max Activating Token Index: 379

Text #18

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5635. Min Act: -0.1700

Data Index: 5178297 (Open Web Text)

Max Activating Token Index: 173

Text #19

Max Range: 1.6954. Min Range: -1.6954

Max Act: 1.5676. Min Act: -0.1447

Data Index: 6249099 (Open Web Text)

Max Activating Token Index: 684