Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 70 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.9083. Min Act: -0.1700

Data Index: 617803 (Open Web Text)

Max Activating Token Index: 419

Text #1

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.8627. Min Act: -0.1700

Data Index: 6175491 (Open Web Text)

Max Activating Token Index: 200

Text #2

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.8517. Min Act: -0.1700

Data Index: 4175373 (Open Web Text)

Max Activating Token Index: 846

Text #3

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.8178. Min Act: -0.1700

Data Index: 2425329 (Open Web Text)

Max Activating Token Index: 358

Text #4

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7690. Min Act: -0.1700

Data Index: 3174464 (Open Web Text)

Max Activating Token Index: 678

Text #5

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7869. Min Act: -0.1700

Data Index: 1189182 (Open Web Text)

Max Activating Token Index: 692

Text #6

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7776. Min Act: -0.1700

Data Index: 3218720 (Open Web Text)

Max Activating Token Index: 403

Text #7

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7628. Min Act: -0.1700

Data Index: 1677590 (Open Web Text)

Max Activating Token Index: 779

Text #8

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7333. Min Act: -0.1700

Data Index: 4644211 (Open Web Text)

Max Activating Token Index: 733

Text #9

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7248. Min Act: -0.1700

Data Index: 4251913 (Open Web Text)

Max Activating Token Index: 915

Text #10

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.6988. Min Act: -0.1700

Data Index: 1419224 (Open Web Text)

Max Activating Token Index: 39

Text #11

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7192. Min Act: -0.1700

Data Index: 8149935 (Open Web Text)

Max Activating Token Index: 816

Text #12

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7066. Min Act: -0.1700

Data Index: 2289918 (Open Web Text)

Max Activating Token Index: 129

Text #13

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7283. Min Act: -0.1700

Data Index: 7152554 (Open Web Text)

Max Activating Token Index: 408

Text #14

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.6810. Min Act: -0.1700

Data Index: 4047648 (Open Web Text)

Max Activating Token Index: 98

Text #15

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.7195. Min Act: -0.1700

Data Index: 5787142 (Open Web Text)

Max Activating Token Index: 558

Text #16

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.6709. Min Act: -0.1700

Data Index: 8464509 (Open Web Text)

Max Activating Token Index: 854

Text #17

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.6715. Min Act: -0.1700

Data Index: 4399568 (Open Web Text)

Max Activating Token Index: 511

Text #18

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.6280. Min Act: -0.1700

Data Index: 4186882 (Open Web Text)

Max Activating Token Index: 726

Text #19

Max Range: 3.9083. Min Range: -3.9083

Max Act: 3.6399. Min Act: -0.1700

Data Index: 3253238 (Open Web Text)

Max Activating Token Index: 64