Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2049 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.3918. Min Act: -0.1700

Data Index: 2765813 (Open Web Text)

Max Activating Token Index: 975

Text #1

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.3362. Min Act: -0.1700

Data Index: 117305 (Open Web Text)

Max Activating Token Index: 593

Text #2

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.2923. Min Act: -0.1700

Data Index: 1134633 (Open Web Text)

Max Activating Token Index: 1007

Text #3

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.2389. Min Act: -0.1700

Data Index: 742376 (Open Web Text)

Max Activating Token Index: 736

Text #4

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.2181. Min Act: -0.1700

Data Index: 1332784 (Open Web Text)

Max Activating Token Index: 314

Text #5

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.2141. Min Act: -0.1700

Data Index: 6472917 (Open Web Text)

Max Activating Token Index: 426

Text #6

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.2289. Min Act: -0.1700

Data Index: 1122411 (Open Web Text)

Max Activating Token Index: 690

Text #7

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.2065. Min Act: -0.1700

Data Index: 7390273 (Open Web Text)

Max Activating Token Index: 99

Text #8

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.2054. Min Act: -0.1700

Data Index: 7625492 (Open Web Text)

Max Activating Token Index: 803

Text #9

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1628. Min Act: -0.1700

Data Index: 538304 (Open Web Text)

Max Activating Token Index: 430

Text #10

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1468. Min Act: -0.1700

Data Index: 6891376 (Open Web Text)

Max Activating Token Index: 778

Text #11

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1641. Min Act: -0.1700

Data Index: 4101026 (Open Web Text)

Max Activating Token Index: 954

Text #12

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1512. Min Act: -0.1700

Data Index: 6184379 (Open Web Text)

Max Activating Token Index: 656

Text #13

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1162. Min Act: -0.1700

Data Index: 6209815 (Open Web Text)

Max Activating Token Index: 632

Text #14

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1190. Min Act: -0.1700

Data Index: 6051870 (Open Web Text)

Max Activating Token Index: 361

Text #15

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1204. Min Act: -0.1700

Data Index: 3362469 (Open Web Text)

Max Activating Token Index: 239

Text #16

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1064. Min Act: -0.1700

Data Index: 5903715 (Open Web Text)

Max Activating Token Index: 329

Text #17

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.0922. Min Act: -0.1700

Data Index: 2774750 (Open Web Text)

Max Activating Token Index: 641

Text #18

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.1103. Min Act: -0.1700

Data Index: 5267626 (Open Web Text)

Max Activating Token Index: 702

Text #19

Max Range: 3.3918. Min Range: -3.3918

Max Act: 3.0681. Min Act: -0.1700

Data Index: 1667343 (Open Web Text)

Max Activating Token Index: 629