Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2344 in Layer 4

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.3815. Min Act: -0.1700

Data Index: 6333882 (Open Web Text)

Max Activating Token Index: 475

Text #1

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.3664. Min Act: -0.1700

Data Index: 6847882 (Open Web Text)

Max Activating Token Index: 526

Text #2

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.3016. Min Act: -0.1700

Data Index: 1288847 (Open Web Text)

Max Activating Token Index: 1014

Text #3

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.3579. Min Act: -0.1700

Data Index: 5694016 (Open Web Text)

Max Activating Token Index: 983

Text #4

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2683. Min Act: -0.1700

Data Index: 7429186 (Open Web Text)

Max Activating Token Index: 507

Text #5

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2870. Min Act: -0.1700

Data Index: 2747289 (Open Web Text)

Max Activating Token Index: 341

Text #6

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2748. Min Act: -0.1700

Data Index: 2522734 (Open Web Text)

Max Activating Token Index: 312

Text #7

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2417. Min Act: -0.1700

Data Index: 1708810 (Open Web Text)

Max Activating Token Index: 910

Text #8

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2674. Min Act: -0.1700

Data Index: 1398451 (Open Web Text)

Max Activating Token Index: 167

Text #9

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2318. Min Act: -0.1700

Data Index: 6484426 (Open Web Text)

Max Activating Token Index: 431

Text #10

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2415. Min Act: -0.1700

Data Index: 5496842 (Open Web Text)

Max Activating Token Index: 488

Text #11

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2290. Min Act: -0.1700

Data Index: 540899 (Open Web Text)

Max Activating Token Index: 547

Text #12

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2290. Min Act: -0.1700

Data Index: 3786569 (Open Web Text)

Max Activating Token Index: 219

Text #13

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2058. Min Act: -0.1700

Data Index: 6822412 (Open Web Text)

Max Activating Token Index: 375

Text #14

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.1840. Min Act: -0.1700

Data Index: 8461074 (Open Web Text)

Max Activating Token Index: 642

Text #15

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.1704. Min Act: -0.1700

Data Index: 2594799 (Open Web Text)

Max Activating Token Index: 328

Text #16

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2133. Min Act: -0.1700

Data Index: 3194831 (Open Web Text)

Max Activating Token Index: 462

Text #17

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2175. Min Act: -0.1700

Data Index: 4841545 (Open Web Text)

Max Activating Token Index: 189

Text #18

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2512. Min Act: -0.1700

Data Index: 2313966 (Open Web Text)

Max Activating Token Index: 541

Text #19

Max Range: 3.3815. Min Range: -3.3815

Max Act: 3.2072. Min Act: -0.1700

Data Index: 2273436 (Open Web Text)

Max Activating Token Index: 455