Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2606 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 3.0112. Min Range: -3.0112

Max Act: 3.0112. Min Act: -0.1700

Data Index: 4611809 (Open Web Text)

Max Activating Token Index: 742

Text #1

Max Range: 3.0112. Min Range: -3.0112

Max Act: 3.0055. Min Act: -0.1700

Data Index: 1887342 (Open Web Text)

Max Activating Token Index: 389

Text #2

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.9826. Min Act: -0.1700

Data Index: 3054294 (Open Web Text)

Max Activating Token Index: 623

Text #3

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.9170. Min Act: -0.1700

Data Index: 563073 (Open Web Text)

Max Activating Token Index: 1023

Text #4

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.9198. Min Act: -0.1700

Data Index: 5873134 (Open Web Text)

Max Activating Token Index: 711

Text #5

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8906. Min Act: -0.1700

Data Index: 7990021 (Open Web Text)

Max Activating Token Index: 691

Text #6

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8814. Min Act: -0.1700

Data Index: 6604351 (Open Web Text)

Max Activating Token Index: 475

Text #7

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8535. Min Act: -0.1700

Data Index: 3838830 (Open Web Text)

Max Activating Token Index: 248

Text #8

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8837. Min Act: -0.1700

Data Index: 8637954 (Open Web Text)

Max Activating Token Index: 839

Text #9

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8671. Min Act: -0.1700

Data Index: 6570761 (Open Web Text)

Max Activating Token Index: 823

Text #10

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8303. Min Act: -0.1700

Data Index: 8582336 (Open Web Text)

Max Activating Token Index: 805

Text #11

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8458. Min Act: -0.1700

Data Index: 5006208 (Open Web Text)

Max Activating Token Index: 300

Text #12

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.7946. Min Act: -0.1700

Data Index: 8659935 (Open Web Text)

Max Activating Token Index: 860

Text #13

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8054. Min Act: -0.1700

Data Index: 8797449 (Open Web Text)

Max Activating Token Index: 1010

Text #14

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8511. Min Act: -0.1700

Data Index: 6250194 (Open Web Text)

Max Activating Token Index: 272

Text #15

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8076. Min Act: -0.1700

Data Index: 1410756 (Open Web Text)

Max Activating Token Index: 386

Text #16

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8161. Min Act: -0.1700

Data Index: 6895719 (Open Web Text)

Max Activating Token Index: 830

Text #17

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8315. Min Act: -0.1700

Data Index: 4569916 (Open Web Text)

Max Activating Token Index: 846

Text #18

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8232. Min Act: -0.1700

Data Index: 3803808 (Open Web Text)

Max Activating Token Index: 349

Text #19

Max Range: 3.0112. Min Range: -3.0112

Max Act: 2.8072. Min Act: -0.1700

Data Index: 6596375 (Open Web Text)

Max Activating Token Index: 899