Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2585 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1218. Min Range: -3.1218

Max Act: 3.1218. Min Act: -0.1700

Data Index: 1910645 (Open Web Text)

Max Activating Token Index: 718

Text #1

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.9905. Min Act: -0.1700

Data Index: 59891 (Open Web Text)

Max Activating Token Index: 249

Text #2

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.9482. Min Act: -0.1700

Data Index: 4812106 (Open Web Text)

Max Activating Token Index: 50

Text #3

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.9454. Min Act: -0.1700

Data Index: 4339070 (Open Web Text)

Max Activating Token Index: 933

Text #4

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.9341. Min Act: -0.1700

Data Index: 6503764 (Open Web Text)

Max Activating Token Index: 634

Text #5

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.9022. Min Act: -0.1700

Data Index: 7151753 (Open Web Text)

Max Activating Token Index: 524

Text #6

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.9131. Min Act: -0.1700

Data Index: 8767052 (Open Web Text)

Max Activating Token Index: 456

Text #7

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.9015. Min Act: -0.1700

Data Index: 447167 (Open Web Text)

Max Activating Token Index: 681

Text #8

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8807. Min Act: -0.1700

Data Index: 3495291 (Open Web Text)

Max Activating Token Index: 1018

Text #9

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8516. Min Act: -0.1700

Data Index: 4697794 (Open Web Text)

Max Activating Token Index: 278

Text #10

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8441. Min Act: -0.1700

Data Index: 4316575 (Open Web Text)

Max Activating Token Index: 149

Text #11

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8518. Min Act: -0.1700

Data Index: 2454237 (Open Web Text)

Max Activating Token Index: 169

Text #12

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8527. Min Act: -0.1700

Data Index: 4874028 (Open Web Text)

Max Activating Token Index: 331

Text #13

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8407. Min Act: -0.1700

Data Index: 5985771 (Open Web Text)

Max Activating Token Index: 1018

Text #14

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8293. Min Act: -0.1700

Data Index: 4716669 (Open Web Text)

Max Activating Token Index: 166

Text #15

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8462. Min Act: -0.1700

Data Index: 4494754 (Open Web Text)

Max Activating Token Index: 439

Text #16

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8217. Min Act: -0.1700

Data Index: 5493287 (Open Web Text)

Max Activating Token Index: 852

Text #17

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8031. Min Act: -0.1700

Data Index: 4861246 (Open Web Text)

Max Activating Token Index: 117

Text #18

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8027. Min Act: -0.1700

Data Index: 1677863 (Open Web Text)

Max Activating Token Index: 241

Text #19

Max Range: 3.1218. Min Range: -3.1218

Max Act: 2.8006. Min Act: -0.1700

Data Index: 2582631 (Open Web Text)

Max Activating Token Index: 38