Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 511 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.4441. Min Act: -0.1700

Data Index: 4112248 (Open Web Text)

Max Activating Token Index: 442

Text #1

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.4204. Min Act: -0.1700

Data Index: 3678345 (Open Web Text)

Max Activating Token Index: 607

Text #2

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.4059. Min Act: -0.1700

Data Index: 7024400 (Open Web Text)

Max Activating Token Index: 106

Text #3

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.3158. Min Act: -0.1700

Data Index: 2887468 (Open Web Text)

Max Activating Token Index: 88

Text #4

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.2830. Min Act: -0.1700

Data Index: 5573332 (Open Web Text)

Max Activating Token Index: 636

Text #5

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.2357. Min Act: -0.1700

Data Index: 7406089 (Open Web Text)

Max Activating Token Index: 881

Text #6

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.2143. Min Act: -0.1700

Data Index: 477462 (Open Web Text)

Max Activating Token Index: 409

Text #7

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.2099. Min Act: -0.1700

Data Index: 2947302 (Open Web Text)

Max Activating Token Index: 177

Text #8

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.2131. Min Act: -0.1700

Data Index: 6327922 (Open Web Text)

Max Activating Token Index: 470

Text #9

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1992. Min Act: -0.1700

Data Index: 190179 (Open Web Text)

Max Activating Token Index: 317

Text #10

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1997. Min Act: -0.1700

Data Index: 1293272 (Open Web Text)

Max Activating Token Index: 934

Text #11

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.2214. Min Act: -0.1700

Data Index: 7206347 (Open Web Text)

Max Activating Token Index: 408

Text #12

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1687. Min Act: -0.1700

Data Index: 7235220 (Open Web Text)

Max Activating Token Index: 334

Text #13

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1765. Min Act: -0.1700

Data Index: 7675037 (Open Web Text)

Max Activating Token Index: 189

Text #14

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1514. Min Act: -0.1700

Data Index: 368699 (Open Web Text)

Max Activating Token Index: 508

Text #15

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1570. Min Act: -0.1700

Data Index: 3688847 (Open Web Text)

Max Activating Token Index: 830

Text #16

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1593. Min Act: -0.1700

Data Index: 1293268 (Open Web Text)

Max Activating Token Index: 287

Text #17

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1245. Min Act: -0.1700

Data Index: 5226379 (Open Web Text)

Max Activating Token Index: 44

Text #18

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1671. Min Act: -0.1700

Data Index: 8069628 (Open Web Text)

Max Activating Token Index: 94

Text #19

Max Range: 3.4441. Min Range: -3.4441

Max Act: 3.1491. Min Act: -0.1700

Data Index: 6337751 (Open Web Text)

Max Activating Token Index: 233