Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 2032 in Layer 3

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.1280. Min Act: -0.1700

Data Index: 1717859 (Open Web Text)

Max Activating Token Index: 566

Text #1

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.0627. Min Act: -0.1700

Data Index: 3684072 (Open Web Text)

Max Activating Token Index: 570

Text #2

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.0054. Min Act: -0.1700

Data Index: 6249310 (Open Web Text)

Max Activating Token Index: 569

Text #3

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.0036. Min Act: -0.1700

Data Index: 8362297 (Open Web Text)

Max Activating Token Index: 566

Text #4

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.0077. Min Act: -0.1700

Data Index: 7276560 (Open Web Text)

Max Activating Token Index: 535

Text #5

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.0199. Min Act: -0.1700

Data Index: 647178 (Open Web Text)

Max Activating Token Index: 868

Text #6

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.0099. Min Act: -0.1700

Data Index: 2106415 (Open Web Text)

Max Activating Token Index: 585

Text #7

Max Range: 5.1280. Min Range: -5.1280

Max Act: 5.0174. Min Act: -0.1700

Data Index: 4213202 (Open Web Text)

Max Activating Token Index: 555

Text #8

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9929. Min Act: -0.1700

Data Index: 8379369 (Open Web Text)

Max Activating Token Index: 565

Text #9

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9840. Min Act: -0.1700

Data Index: 888456 (Open Web Text)

Max Activating Token Index: 283

Text #10

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9881. Min Act: -0.1700

Data Index: 3016169 (Open Web Text)

Max Activating Token Index: 314

Text #11

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9799. Min Act: -0.1700

Data Index: 6736033 (Open Web Text)

Max Activating Token Index: 621

Text #12

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9843. Min Act: -0.1700

Data Index: 6000933 (Open Web Text)

Max Activating Token Index: 612

Text #13

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9943. Min Act: -0.1700

Data Index: 1655259 (Open Web Text)

Max Activating Token Index: 817

Text #14

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9599. Min Act: -0.1700

Data Index: 3936686 (Open Web Text)

Max Activating Token Index: 579

Text #15

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9840. Min Act: -0.1700

Data Index: 8740271 (Open Web Text)

Max Activating Token Index: 759

Text #16

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9769. Min Act: -0.1700

Data Index: 8413930 (Open Web Text)

Max Activating Token Index: 520

Text #17

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9491. Min Act: -0.1700

Data Index: 3785101 (Open Web Text)

Max Activating Token Index: 223

Text #18

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9045. Min Act: -0.1700

Data Index: 862923 (Open Web Text)

Max Activating Token Index: 907

Text #19

Max Range: 5.1280. Min Range: -5.1280

Max Act: 4.9661. Min Act: -0.1700

Data Index: 974513 (Open Web Text)

Max Activating Token Index: 789