Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2178 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.3419. Min Act: -0.1700

Data Index: 2455600 (Open Web Text)

Max Activating Token Index: 1021

Text #1

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.2452. Min Act: -0.1700

Data Index: 6182717 (Open Web Text)

Max Activating Token Index: 219

Text #2

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.2437. Min Act: -0.1700

Data Index: 5459025 (Open Web Text)

Max Activating Token Index: 732

Text #3

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.2312. Min Act: -0.1700

Data Index: 4428538 (Open Web Text)

Max Activating Token Index: 844

Text #4

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.2042. Min Act: -0.1700

Data Index: 2676147 (Open Web Text)

Max Activating Token Index: 146

Text #5

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1976. Min Act: -0.1700

Data Index: 4802120 (Open Web Text)

Max Activating Token Index: 908

Text #6

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.2111. Min Act: -0.1700

Data Index: 1743459 (Open Web Text)

Max Activating Token Index: 242

Text #7

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1853. Min Act: -0.1700

Data Index: 8225724 (Open Web Text)

Max Activating Token Index: 817

Text #8

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1593. Min Act: -0.1700

Data Index: 8068049 (Open Web Text)

Max Activating Token Index: 761

Text #9

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1866. Min Act: -0.1700

Data Index: 841694 (Open Web Text)

Max Activating Token Index: 833

Text #10

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1865. Min Act: -0.1700

Data Index: 4550544 (Open Web Text)

Max Activating Token Index: 307

Text #11

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1584. Min Act: -0.1700

Data Index: 3481917 (Open Web Text)

Max Activating Token Index: 984

Text #12

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1475. Min Act: -0.1700

Data Index: 8502802 (Open Web Text)

Max Activating Token Index: 610

Text #13

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1657. Min Act: -0.1700

Data Index: 2315238 (Open Web Text)

Max Activating Token Index: 948

Text #14

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1835. Min Act: -0.1700

Data Index: 3400085 (Open Web Text)

Max Activating Token Index: 347

Text #15

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1668. Min Act: -0.1700

Data Index: 6463204 (Open Web Text)

Max Activating Token Index: 113

Text #16

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1391. Min Act: -0.1700

Data Index: 2627879 (Open Web Text)

Max Activating Token Index: 728

Text #17

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1532. Min Act: -0.1700

Data Index: 407332 (Open Web Text)

Max Activating Token Index: 416

Text #18

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1265. Min Act: -0.1700

Data Index: 4362813 (Open Web Text)

Max Activating Token Index: 492

Text #19

Max Range: 3.3419. Min Range: -3.3419

Max Act: 3.1386. Min Act: -0.1700

Data Index: 78147 (Open Web Text)

Max Activating Token Index: 472