Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1664 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.6576. Min Act: -0.1700

Data Index: 6733508 (Open Web Text)

Max Activating Token Index: 857

Text #1

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.6336. Min Act: -0.1700

Data Index: 3548618 (Open Web Text)

Max Activating Token Index: 118

Text #2

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.6440. Min Act: -0.1700

Data Index: 7686070 (Open Web Text)

Max Activating Token Index: 524

Text #3

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5893. Min Act: -0.1700

Data Index: 3478930 (Open Web Text)

Max Activating Token Index: 859

Text #4

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5910. Min Act: -0.1700

Data Index: 8341136 (Open Web Text)

Max Activating Token Index: 166

Text #5

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5441. Min Act: -0.1700

Data Index: 4798506 (Open Web Text)

Max Activating Token Index: 445

Text #6

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5739. Min Act: -0.1700

Data Index: 724205 (Open Web Text)

Max Activating Token Index: 945

Text #7

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5753. Min Act: -0.1700

Data Index: 6534790 (Open Web Text)

Max Activating Token Index: 818

Text #8

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5731. Min Act: -0.1700

Data Index: 6699448 (Open Web Text)

Max Activating Token Index: 830

Text #9

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5837. Min Act: -0.1700

Data Index: 6984447 (Open Web Text)

Max Activating Token Index: 468

Text #10

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5603. Min Act: -0.1700

Data Index: 7246772 (Open Web Text)

Max Activating Token Index: 425

Text #11

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5407. Min Act: -0.1700

Data Index: 5307914 (Open Web Text)

Max Activating Token Index: 1019

Text #12

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5552. Min Act: -0.1700

Data Index: 3556096 (Open Web Text)

Max Activating Token Index: 536

Text #13

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5250. Min Act: -0.1700

Data Index: 2128502 (Open Web Text)

Max Activating Token Index: 440

Text #14

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5102. Min Act: -0.1700

Data Index: 839783 (Open Web Text)

Max Activating Token Index: 111

Text #15

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5056. Min Act: -0.1700

Data Index: 3700458 (Open Web Text)

Max Activating Token Index: 1012

Text #16

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5054. Min Act: -0.1700

Data Index: 7678029 (Open Web Text)

Max Activating Token Index: 520

Text #17

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5244. Min Act: -0.1700

Data Index: 764620 (Open Web Text)

Max Activating Token Index: 998

Text #18

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5264. Min Act: -0.1700

Data Index: 2147806 (Open Web Text)

Max Activating Token Index: 215

Text #19

Max Range: 3.6576. Min Range: -3.6576

Max Act: 3.5353. Min Act: -0.1700

Data Index: 4502835 (Open Web Text)

Max Activating Token Index: 827