Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 1212 in Layer 2

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.6450. Min Act: -0.1700

Data Index: 8529259 (Open Web Text)

Max Activating Token Index: 403

Text #1

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5525. Min Act: -0.1700

Data Index: 4980675 (Open Web Text)

Max Activating Token Index: 886

Text #2

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5500. Min Act: -0.1700

Data Index: 5565350 (Open Web Text)

Max Activating Token Index: 712

Text #3

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5079. Min Act: -0.1700

Data Index: 8271289 (Open Web Text)

Max Activating Token Index: 309

Text #4

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5037. Min Act: -0.1700

Data Index: 1025864 (Open Web Text)

Max Activating Token Index: 969

Text #5

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5235. Min Act: -0.1700

Data Index: 1024382 (Open Web Text)

Max Activating Token Index: 390

Text #6

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5291. Min Act: -0.1700

Data Index: 7220249 (Open Web Text)

Max Activating Token Index: 121

Text #7

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4837. Min Act: -0.1700

Data Index: 321018 (Open Web Text)

Max Activating Token Index: 878

Text #8

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4759. Min Act: -0.1700

Data Index: 7188464 (Open Web Text)

Max Activating Token Index: 137

Text #9

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4808. Min Act: -0.1700

Data Index: 2782172 (Open Web Text)

Max Activating Token Index: 981

Text #10

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.5113. Min Act: -0.1700

Data Index: 75967 (Open Web Text)

Max Activating Token Index: 938

Text #11

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4776. Min Act: -0.1700

Data Index: 8699024 (Open Web Text)

Max Activating Token Index: 356

Text #12

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4306. Min Act: -0.1700

Data Index: 3458880 (Open Web Text)

Max Activating Token Index: 446

Text #13

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4538. Min Act: -0.1700

Data Index: 7025057 (Open Web Text)

Max Activating Token Index: 455

Text #14

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4879. Min Act: -0.1700

Data Index: 7242355 (Open Web Text)

Max Activating Token Index: 738

Text #15

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4605. Min Act: -0.1700

Data Index: 2451966 (Open Web Text)

Max Activating Token Index: 464

Text #16

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4342. Min Act: -0.1700

Data Index: 8114082 (Open Web Text)

Max Activating Token Index: 504

Text #17

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4476. Min Act: -0.1700

Data Index: 1100267 (Open Web Text)

Max Activating Token Index: 798

Text #18

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.4297. Min Act: -0.1700

Data Index: 7563770 (Open Web Text)

Max Activating Token Index: 631

Text #19

Max Range: 4.6450. Min Range: -4.6450

Max Act: 4.3965. Min Act: -0.1700

Data Index: 2109609 (Open Web Text)

Max Activating Token Index: 460