Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 328 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.3218. Min Range: -5.3218

Max Act: 5.3218. Min Act: -0.1700

Data Index: 3116830 (Open Web Text)

Max Activating Token Index: 314

Text #1

Max Range: 5.3218. Min Range: -5.3218

Max Act: 5.0608. Min Act: -0.1700

Data Index: 5026389 (Open Web Text)

Max Activating Token Index: 813

Text #2

Max Range: 5.3218. Min Range: -5.3218

Max Act: 5.0132. Min Act: -0.1700

Data Index: 7211787 (Open Web Text)

Max Activating Token Index: 896

Text #3

Max Range: 5.3218. Min Range: -5.3218

Max Act: 5.0013. Min Act: -0.1700

Data Index: 6844914 (Open Web Text)

Max Activating Token Index: 317

Text #4

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.9794. Min Act: -0.1700

Data Index: 5180742 (Open Web Text)

Max Activating Token Index: 956

Text #5

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.9553. Min Act: -0.1700

Data Index: 6960111 (Open Web Text)

Max Activating Token Index: 430

Text #6

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.9392. Min Act: -0.1700

Data Index: 3829137 (Open Web Text)

Max Activating Token Index: 18

Text #7

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.9494. Min Act: -0.1700

Data Index: 8769944 (Open Web Text)

Max Activating Token Index: 815

Text #8

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.9345. Min Act: -0.1700

Data Index: 5289351 (Open Web Text)

Max Activating Token Index: 366

Text #9

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.9024. Min Act: -0.1700

Data Index: 5696379 (Open Web Text)

Max Activating Token Index: 211

Text #10

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.9492. Min Act: -0.1700

Data Index: 6769962 (Open Web Text)

Max Activating Token Index: 275

Text #11

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8604. Min Act: -0.1700

Data Index: 2040663 (Open Web Text)

Max Activating Token Index: 152

Text #12

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8421. Min Act: -0.1700

Data Index: 7452905 (Open Web Text)

Max Activating Token Index: 446

Text #13

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8557. Min Act: -0.1700

Data Index: 5022505 (Open Web Text)

Max Activating Token Index: 846

Text #14

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8921. Min Act: -0.1700

Data Index: 5530068 (Open Web Text)

Max Activating Token Index: 813

Text #15

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8884. Min Act: -0.1700

Data Index: 5975100 (Open Web Text)

Max Activating Token Index: 580

Text #16

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8568. Min Act: -0.1700

Data Index: 3020628 (Open Web Text)

Max Activating Token Index: 851

Text #17

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8810. Min Act: -0.1700

Data Index: 6320150 (Open Web Text)

Max Activating Token Index: 442

Text #18

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8424. Min Act: -0.1700

Data Index: 2309109 (Open Web Text)

Max Activating Token Index: 260

Text #19

Max Range: 5.3218. Min Range: -5.3218

Max Act: 4.8116. Min Act: -0.1700

Data Index: 3859186 (Open Web Text)

Max Activating Token Index: 820