Model: SoLU Model: 8 Layers, 4096 Neurons per Layer

Dataset: The Pile

Neuron 2325 in Layer 6

Load this data into an Interactive Neuroscope

See Documentation here

Transformer Lens Loading: HookedTransformer.from_pretrained('solu-8l-pile')

Text #0

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.6758. Min Act: -0.0000

Data Index: 1615060 (The Pile)

Max Activating Token Index: 484

Text #1

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.6061. Min Act: -0.0000

Data Index: 1778418 (The Pile)

Max Activating Token Index: 634

Text #2

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.3928. Min Act: -0.0000

Data Index: 5944 (The Pile)

Max Activating Token Index: 510

Text #3

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.3096. Min Act: -0.0000

Data Index: 1700384 (The Pile)

Max Activating Token Index: 857

Text #4

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.3046. Min Act: -0.0000

Data Index: 861444 (The Pile)

Max Activating Token Index: 489

Text #5

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1974. Min Act: -0.0000

Data Index: 70936 (The Pile)

Max Activating Token Index: 964

Text #6

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1963. Min Act: -0.0000

Data Index: 1601988 (The Pile)

Max Activating Token Index: 649

Text #7

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.2436. Min Act: -0.0000

Data Index: 1998608 (The Pile)

Max Activating Token Index: 279

Text #8

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.2128. Min Act: -0.0000

Data Index: 909104 (The Pile)

Max Activating Token Index: 202

Text #9

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1922. Min Act: -0.0000

Data Index: 1047928 (The Pile)

Max Activating Token Index: 945

Text #10

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.2178. Min Act: -0.0000

Data Index: 1855363 (The Pile)

Max Activating Token Index: 106

Text #11

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1695. Min Act: -0.0000

Data Index: 237374 (The Pile)

Max Activating Token Index: 230

Text #12

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1313. Min Act: -0.0000

Data Index: 1895137 (The Pile)

Max Activating Token Index: 592

Text #13

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.0907. Min Act: -0.0000

Data Index: 627546 (The Pile)

Max Activating Token Index: 574

Text #14

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.0836. Min Act: -0.0000

Data Index: 1816003 (The Pile)

Max Activating Token Index: 567

Text #15

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1115. Min Act: -0.0000

Data Index: 1156644 (The Pile)

Max Activating Token Index: 301

Text #16

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1763. Min Act: -0.0000

Data Index: 75651 (The Pile)

Max Activating Token Index: 917

Text #17

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1404. Min Act: 0.0000

Data Index: 799748 (The Pile)

Max Activating Token Index: 953

Text #18

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1133. Min Act: -0.0000

Data Index: 1372821 (The Pile)

Max Activating Token Index: 538

Text #19

Max Range: 1.6758. Min Range: -1.6758

Max Act: 1.1083. Min Act: -0.0000

Data Index: 1718270 (The Pile)

Max Activating Token Index: 748