Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 593 in Layer 5

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5985. Min Act: -0.1700

Data Index: 710278 (Open Web Text)

Max Activating Token Index: 328

Text #1

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5938. Min Act: -0.1700

Data Index: 8026041 (Open Web Text)

Max Activating Token Index: 34

Text #2

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5317. Min Act: -0.1700

Data Index: 5522793 (Open Web Text)

Max Activating Token Index: 457

Text #3

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5023. Min Act: -0.1700

Data Index: 2541009 (Open Web Text)

Max Activating Token Index: 960

Text #4

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5286. Min Act: -0.1700

Data Index: 2645511 (Open Web Text)

Max Activating Token Index: 869

Text #5

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4924. Min Act: -0.1700

Data Index: 4628482 (Open Web Text)

Max Activating Token Index: 237

Text #6

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5091. Min Act: -0.1700

Data Index: 4256566 (Open Web Text)

Max Activating Token Index: 47

Text #7

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5119. Min Act: -0.1700

Data Index: 2085000 (Open Web Text)

Max Activating Token Index: 424

Text #8

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5236. Min Act: -0.1700

Data Index: 129784 (Open Web Text)

Max Activating Token Index: 469

Text #9

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5018. Min Act: -0.1700

Data Index: 3578467 (Open Web Text)

Max Activating Token Index: 950

Text #10

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5127. Min Act: -0.1700

Data Index: 7984175 (Open Web Text)

Max Activating Token Index: 249

Text #11

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5111. Min Act: -0.1700

Data Index: 7394091 (Open Web Text)

Max Activating Token Index: 359

Text #12

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4949. Min Act: -0.1700

Data Index: 3210797 (Open Web Text)

Max Activating Token Index: 759

Text #13

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.5117. Min Act: -0.1700

Data Index: 6052465 (Open Web Text)

Max Activating Token Index: 86

Text #14

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4929. Min Act: -0.1700

Data Index: 1013280 (Open Web Text)

Max Activating Token Index: 874

Text #15

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4757. Min Act: -0.1700

Data Index: 1717983 (Open Web Text)

Max Activating Token Index: 574

Text #16

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4925. Min Act: -0.1700

Data Index: 998931 (Open Web Text)

Max Activating Token Index: 1001

Text #17

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4768. Min Act: -0.1700

Data Index: 4435370 (Open Web Text)

Max Activating Token Index: 512

Text #18

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4987. Min Act: -0.1700

Data Index: 809268 (Open Web Text)

Max Activating Token Index: 898

Text #19

Max Range: 4.5985. Min Range: -4.5985

Max Act: 4.4830. Min Act: -0.1700

Data Index: 1661976 (Open Web Text)

Max Activating Token Index: 286