Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2577 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0931. Min Act: -0.1700

Data Index: 7214119 (Open Web Text)

Max Activating Token Index: 371

Text #1

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0271. Min Act: -0.1700

Data Index: 5137146 (Open Web Text)

Max Activating Token Index: 684

Text #2

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0230. Min Act: -0.1700

Data Index: 7214117 (Open Web Text)

Max Activating Token Index: 112

Text #3

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0324. Min Act: -0.1700

Data Index: 6829706 (Open Web Text)

Max Activating Token Index: 200

Text #4

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0409. Min Act: -0.1700

Data Index: 6737999 (Open Web Text)

Max Activating Token Index: 15

Text #5

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0099. Min Act: -0.1700

Data Index: 8121720 (Open Web Text)

Max Activating Token Index: 615

Text #6

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0053. Min Act: -0.1700

Data Index: 2131822 (Open Web Text)

Max Activating Token Index: 300

Text #7

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0139. Min Act: -0.1700

Data Index: 7224316 (Open Web Text)

Max Activating Token Index: 228

Text #8

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0133. Min Act: -0.1700

Data Index: 5785576 (Open Web Text)

Max Activating Token Index: 355

Text #9

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0050. Min Act: -0.1700

Data Index: 1500976 (Open Web Text)

Max Activating Token Index: 282

Text #10

Max Range: 4.0931. Min Range: -4.0931

Max Act: 4.0092. Min Act: -0.1700

Data Index: 6480034 (Open Web Text)

Max Activating Token Index: 624

Text #11

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9909. Min Act: -0.1700

Data Index: 7245012 (Open Web Text)

Max Activating Token Index: 343

Text #12

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9888. Min Act: -0.1700

Data Index: 6397336 (Open Web Text)

Max Activating Token Index: 449

Text #13

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9664. Min Act: -0.1700

Data Index: 2809311 (Open Web Text)

Max Activating Token Index: 539

Text #14

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9707. Min Act: -0.1700

Data Index: 6397327 (Open Web Text)

Max Activating Token Index: 404

Text #15

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9935. Min Act: -0.1700

Data Index: 6069638 (Open Web Text)

Max Activating Token Index: 615

Text #16

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9721. Min Act: -0.1700

Data Index: 532746 (Open Web Text)

Max Activating Token Index: 113

Text #17

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9709. Min Act: -0.1700

Data Index: 2645823 (Open Web Text)

Max Activating Token Index: 724

Text #18

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9693. Min Act: -0.1700

Data Index: 5379764 (Open Web Text)

Max Activating Token Index: 125

Text #19

Max Range: 4.0931. Min Range: -4.0931

Max Act: 3.9600. Min Act: -0.1700

Data Index: 2052027 (Open Web Text)

Max Activating Token Index: 245