Model: GPT-2 Small: 12 Layers, 3072 Neurons per Layer

Dataset: Open Web Text

Neuron 620 in Layer 1

Load this data into an Interactive Neuroscope

See Documentation here

Hooked Transformer Loading: HookedTransformer.from_pretrained('gpt2-small')

Click the toggle to see the full text!

Text #0

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1973. Min Act: -0.1700

Data Index: 5892031 (Open Web Text)

Max Activating Token Index: 575

Text #1

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1952. Min Act: -0.1700

Data Index: 2688848 (Open Web Text)

Max Activating Token Index: 236

Text #2

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.2110. Min Act: -0.1700

Data Index: 1896416 (Open Web Text)

Max Activating Token Index: 242

Text #3

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1967. Min Act: -0.1700

Data Index: 212898 (Open Web Text)

Max Activating Token Index: 476

Text #4

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1750. Min Act: -0.1700

Data Index: 2540003 (Open Web Text)

Max Activating Token Index: 541

Text #5

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1212. Min Act: -0.1700

Data Index: 8122417 (Open Web Text)

Max Activating Token Index: 167

Text #6

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1056. Min Act: -0.1700

Data Index: 2500314 (Open Web Text)

Max Activating Token Index: 101

Text #7

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1127. Min Act: -0.1700

Data Index: 5860582 (Open Web Text)

Max Activating Token Index: 72

Text #8

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1012. Min Act: -0.1700

Data Index: 5280253 (Open Web Text)

Max Activating Token Index: 645

Text #9

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.1083. Min Act: -0.1700

Data Index: 2032599 (Open Web Text)

Max Activating Token Index: 425

Text #10

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0899. Min Act: -0.1700

Data Index: 102201 (Open Web Text)

Max Activating Token Index: 198

Text #11

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0688. Min Act: -0.1700

Data Index: 606381 (Open Web Text)

Max Activating Token Index: 530

Text #12

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0674. Min Act: -0.1700

Data Index: 2863973 (Open Web Text)

Max Activating Token Index: 842

Text #13

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0789. Min Act: -0.1700

Data Index: 4512966 (Open Web Text)

Max Activating Token Index: 370

Text #14

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0832. Min Act: -0.1700

Data Index: 4590102 (Open Web Text)

Max Activating Token Index: 218

Text #15

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0791. Min Act: -0.1700

Data Index: 3198703 (Open Web Text)

Max Activating Token Index: 214

Text #16

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0825. Min Act: -0.1700

Data Index: 1793409 (Open Web Text)

Max Activating Token Index: 81

Text #17

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0651. Min Act: -0.1700

Data Index: 655261 (Open Web Text)

Max Activating Token Index: 352

Text #18

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0916. Min Act: -0.1700

Data Index: 2811175 (Open Web Text)

Max Activating Token Index: 452

Text #19

Max Range: 4.2110. Min Range: -4.2110

Max Act: 4.0615. Min Act: -0.1700

Data Index: 986152 (Open Web Text)

Max Activating Token Index: 47