Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 590 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.2435. Min Act: -0.1700

Data Index: 6963628 (Open Web Text)

Max Activating Token Index: 581

Text #1

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.1987. Min Act: -0.1700

Data Index: 5310580 (Open Web Text)

Max Activating Token Index: 68

Text #2

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0991. Min Act: -0.1700

Data Index: 5869170 (Open Web Text)

Max Activating Token Index: 930

Text #3

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0804. Min Act: -0.1700

Data Index: 4054961 (Open Web Text)

Max Activating Token Index: 562

Text #4

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0523. Min Act: -0.1700

Data Index: 6912801 (Open Web Text)

Max Activating Token Index: 407

Text #5

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0569. Min Act: -0.1700

Data Index: 7024618 (Open Web Text)

Max Activating Token Index: 133

Text #6

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0821. Min Act: -0.1700

Data Index: 7513523 (Open Web Text)

Max Activating Token Index: 943

Text #7

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0150. Min Act: -0.1700

Data Index: 6339218 (Open Web Text)

Max Activating Token Index: 226

Text #8

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0296. Min Act: -0.1700

Data Index: 6654692 (Open Web Text)

Max Activating Token Index: 407

Text #9

Max Range: 5.2435. Min Range: -5.2435

Max Act: 5.0071. Min Act: -0.1700

Data Index: 8059817 (Open Web Text)

Max Activating Token Index: 486

Text #10

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9960. Min Act: -0.1700

Data Index: 7702418 (Open Web Text)

Max Activating Token Index: 592

Text #11

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9424. Min Act: -0.1700

Data Index: 5373153 (Open Web Text)

Max Activating Token Index: 547

Text #12

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9310. Min Act: -0.1700

Data Index: 5443194 (Open Web Text)

Max Activating Token Index: 694

Text #13

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9422. Min Act: -0.1700

Data Index: 5365214 (Open Web Text)

Max Activating Token Index: 633

Text #14

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9544. Min Act: -0.1700

Data Index: 7629328 (Open Web Text)

Max Activating Token Index: 690

Text #15

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9427. Min Act: -0.1700

Data Index: 6055869 (Open Web Text)

Max Activating Token Index: 846

Text #16

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9525. Min Act: -0.1700

Data Index: 5354594 (Open Web Text)

Max Activating Token Index: 985

Text #17

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9708. Min Act: -0.1700

Data Index: 6000112 (Open Web Text)

Max Activating Token Index: 550

Text #18

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9643. Min Act: -0.1700

Data Index: 8334616 (Open Web Text)

Max Activating Token Index: 453

Text #19

Max Range: 5.2435. Min Range: -5.2435

Max Act: 4.9500. Min Act: -0.1700

Data Index: 4455059 (Open Web Text)

Max Activating Token Index: 138