Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2809 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.1297. Min Act: -0.1700

Data Index: 4919161 (Open Web Text)

Max Activating Token Index: 1018

Text #1

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.1502. Min Act: -0.1700

Data Index: 5565971 (Open Web Text)

Max Activating Token Index: 364

Text #2

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.1075. Min Act: -0.1700

Data Index: 6412638 (Open Web Text)

Max Activating Token Index: 583

Text #3

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.1073. Min Act: -0.1700

Data Index: 1743355 (Open Web Text)

Max Activating Token Index: 491

Text #4

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0939. Min Act: -0.1700

Data Index: 6267068 (Open Web Text)

Max Activating Token Index: 727

Text #5

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0918. Min Act: -0.1700

Data Index: 3702171 (Open Web Text)

Max Activating Token Index: 640

Text #6

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0643. Min Act: -0.1700

Data Index: 1863422 (Open Web Text)

Max Activating Token Index: 1022

Text #7

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0941. Min Act: -0.1700

Data Index: 5597137 (Open Web Text)

Max Activating Token Index: 339

Text #8

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0857. Min Act: -0.1700

Data Index: 7170077 (Open Web Text)

Max Activating Token Index: 821

Text #9

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0594. Min Act: -0.1700

Data Index: 7322200 (Open Web Text)

Max Activating Token Index: 520

Text #10

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0628. Min Act: -0.1700

Data Index: 7695235 (Open Web Text)

Max Activating Token Index: 1001

Text #11

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0295. Min Act: -0.1700

Data Index: 667646 (Open Web Text)

Max Activating Token Index: 803

Text #12

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0408. Min Act: -0.1700

Data Index: 4771298 (Open Web Text)

Max Activating Token Index: 968

Text #13

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0182. Min Act: -0.1700

Data Index: 7990979 (Open Web Text)

Max Activating Token Index: 72

Text #14

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0093. Min Act: -0.1700

Data Index: 1957024 (Open Web Text)

Max Activating Token Index: 20

Text #15

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0009. Min Act: -0.1700

Data Index: 7923466 (Open Web Text)

Max Activating Token Index: 526

Text #16

Max Range: 3.1502. Min Range: -3.1502

Max Act: 2.9916. Min Act: -0.1700

Data Index: 3371328 (Open Web Text)

Max Activating Token Index: 12

Text #17

Max Range: 3.1502. Min Range: -3.1502

Max Act: 2.9989. Min Act: -0.1700

Data Index: 5338968 (Open Web Text)

Max Activating Token Index: 588

Text #18

Max Range: 3.1502. Min Range: -3.1502

Max Act: 3.0138. Min Act: -0.1700

Data Index: 1959752 (Open Web Text)

Max Activating Token Index: 439

Text #19

Max Range: 3.1502. Min Range: -3.1502

Max Act: 2.9982. Min Act: -0.1700

Data Index: 1897510 (Open Web Text)

Max Activating Token Index: 143