Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 778 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.9835. Min Act: -0.1700

Data Index: 8036509 (Open Web Text)

Max Activating Token Index: 510

Text #1

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.7614. Min Act: -0.1700

Data Index: 771178 (Open Web Text)

Max Activating Token Index: 630

Text #2

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.6197. Min Act: -0.1700

Data Index: 2296351 (Open Web Text)

Max Activating Token Index: 96

Text #3

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.6852. Min Act: -0.1700

Data Index: 4498572 (Open Web Text)

Max Activating Token Index: 816

Text #4

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.6723. Min Act: -0.1700

Data Index: 3517756 (Open Web Text)

Max Activating Token Index: 167

Text #5

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.6569. Min Act: -0.1700

Data Index: 8338974 (Open Web Text)

Max Activating Token Index: 737

Text #6

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.6005. Min Act: -0.1700

Data Index: 7058771 (Open Web Text)

Max Activating Token Index: 180

Text #7

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5795. Min Act: -0.1700

Data Index: 3574644 (Open Web Text)

Max Activating Token Index: 483

Text #8

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5395. Min Act: -0.1700

Data Index: 2974487 (Open Web Text)

Max Activating Token Index: 510

Text #9

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5274. Min Act: -0.1700

Data Index: 685309 (Open Web Text)

Max Activating Token Index: 414

Text #10

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.4905. Min Act: -0.1700

Data Index: 2649825 (Open Web Text)

Max Activating Token Index: 487

Text #11

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5294. Min Act: -0.1700

Data Index: 6633751 (Open Web Text)

Max Activating Token Index: 364

Text #12

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5301. Min Act: -0.1700

Data Index: 1807519 (Open Web Text)

Max Activating Token Index: 529

Text #13

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5140. Min Act: -0.1700

Data Index: 6990998 (Open Web Text)

Max Activating Token Index: 184

Text #14

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5107. Min Act: -0.1700

Data Index: 5882777 (Open Web Text)

Max Activating Token Index: 875

Text #15

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.5224. Min Act: -0.1700

Data Index: 7436811 (Open Web Text)

Max Activating Token Index: 526

Text #16

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.4453. Min Act: -0.1700

Data Index: 253282 (Open Web Text)

Max Activating Token Index: 667

Text #17

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.4680. Min Act: -0.1700

Data Index: 976216 (Open Web Text)

Max Activating Token Index: 271

Text #18

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.4401. Min Act: -0.1700

Data Index: 5136977 (Open Web Text)

Max Activating Token Index: 959

Text #19

Max Range: 4.9835. Min Range: -4.9835

Max Act: 4.4465. Min Act: -0.1700

Data Index: 2001083 (Open Web Text)

Max Activating Token Index: 510