Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2807 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.1287. Min Act: -0.1700

Data Index: 989821 (Open Web Text)

Max Activating Token Index: 71

Text #1

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.1108. Min Act: -0.1700

Data Index: 1386562 (Open Web Text)

Max Activating Token Index: 586

Text #2

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.1049. Min Act: -0.1700

Data Index: 3214759 (Open Web Text)

Max Activating Token Index: 40

Text #3

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0849. Min Act: -0.1700

Data Index: 7704377 (Open Web Text)

Max Activating Token Index: 529

Text #4

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.1025. Min Act: -0.1700

Data Index: 1364388 (Open Web Text)

Max Activating Token Index: 549

Text #5

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0466. Min Act: -0.1700

Data Index: 1519589 (Open Web Text)

Max Activating Token Index: 62

Text #6

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0538. Min Act: -0.1700

Data Index: 4189574 (Open Web Text)

Max Activating Token Index: 417

Text #7

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0450. Min Act: -0.1700

Data Index: 4306970 (Open Web Text)

Max Activating Token Index: 655

Text #8

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0348. Min Act: -0.1700

Data Index: 3983033 (Open Web Text)

Max Activating Token Index: 286

Text #9

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0533. Min Act: -0.1700

Data Index: 2692788 (Open Web Text)

Max Activating Token Index: 322

Text #10

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0529. Min Act: -0.1700

Data Index: 2161165 (Open Web Text)

Max Activating Token Index: 32

Text #11

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0569. Min Act: -0.1700

Data Index: 8390701 (Open Web Text)

Max Activating Token Index: 496

Text #12

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0428. Min Act: -0.1700

Data Index: 6978458 (Open Web Text)

Max Activating Token Index: 69

Text #13

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0386. Min Act: -0.1700

Data Index: 6460350 (Open Web Text)

Max Activating Token Index: 359

Text #14

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0353. Min Act: -0.1700

Data Index: 5785629 (Open Web Text)

Max Activating Token Index: 546

Text #15

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0247. Min Act: -0.1700

Data Index: 4458204 (Open Web Text)

Max Activating Token Index: 798

Text #16

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0429. Min Act: -0.1700

Data Index: 1274645 (Open Web Text)

Max Activating Token Index: 283

Text #17

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0175. Min Act: -0.1700

Data Index: 4161903 (Open Web Text)

Max Activating Token Index: 252

Text #18

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0161. Min Act: -0.1700

Data Index: 2646819 (Open Web Text)

Max Activating Token Index: 288

Text #19

Max Range: 4.1287. Min Range: -4.1287

Max Act: 4.0164. Min Act: -0.1700

Data Index: 148577 (Open Web Text)

Max Activating Token Index: 480