Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2401 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0272. Min Range: -3.0272

Max Act: 3.0272. Min Act: -0.1700

Data Index: 7385022 (Open Web Text)

Max Activating Token Index: 298

Text #1

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.9483. Min Act: -0.1700

Data Index: 7136031 (Open Web Text)

Max Activating Token Index: 537

Text #2

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.9375. Min Act: -0.1700

Data Index: 5521565 (Open Web Text)

Max Activating Token Index: 427

Text #3

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.9402. Min Act: -0.1700

Data Index: 1400181 (Open Web Text)

Max Activating Token Index: 606

Text #4

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.9091. Min Act: -0.1700

Data Index: 3601935 (Open Web Text)

Max Activating Token Index: 352

Text #5

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.9166. Min Act: -0.1700

Data Index: 660794 (Open Web Text)

Max Activating Token Index: 424

Text #6

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8977. Min Act: -0.1700

Data Index: 3727591 (Open Web Text)

Max Activating Token Index: 463

Text #7

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8953. Min Act: -0.1700

Data Index: 7535454 (Open Web Text)

Max Activating Token Index: 533

Text #8

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.9023. Min Act: -0.1700

Data Index: 3250986 (Open Web Text)

Max Activating Token Index: 536

Text #9

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.9032. Min Act: -0.1700

Data Index: 6581351 (Open Web Text)

Max Activating Token Index: 689

Text #10

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8940. Min Act: -0.1700

Data Index: 67993 (Open Web Text)

Max Activating Token Index: 482

Text #11

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8833. Min Act: -0.1700

Data Index: 141350 (Open Web Text)

Max Activating Token Index: 308

Text #12

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8568. Min Act: -0.1700

Data Index: 252613 (Open Web Text)

Max Activating Token Index: 149

Text #13

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8709. Min Act: -0.1700

Data Index: 1583672 (Open Web Text)

Max Activating Token Index: 762

Text #14

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8528. Min Act: -0.1700

Data Index: 2506127 (Open Web Text)

Max Activating Token Index: 510

Text #15

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8593. Min Act: -0.1700

Data Index: 8340524 (Open Web Text)

Max Activating Token Index: 310

Text #16

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8572. Min Act: -0.1700

Data Index: 4847383 (Open Web Text)

Max Activating Token Index: 450

Text #17

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8630. Min Act: -0.1700

Data Index: 211039 (Open Web Text)

Max Activating Token Index: 431

Text #18

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8367. Min Act: -0.1700

Data Index: 6926682 (Open Web Text)

Max Activating Token Index: 711

Text #19

Max Range: 3.0272. Min Range: -3.0272

Max Act: 2.8353. Min Act: -0.1700

Data Index: 5559627 (Open Web Text)

Max Activating Token Index: 117